{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Goal - show examples of cross-validation for model selection and bootstrapping for prediction error estimation.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix, roc_auc_score\n",
    "from sklearn import linear_model\n",
    "import os\n",
    "import course_utils as bd\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "%matplotlib inline\n",
    "import imp\n",
    "imp.reload(bd)\n",
    "\n",
    "#We assume data is in a parallel directory to this one called 'data'\n",
    "cwd = os.getcwd()\n",
    "datadir = '/'.join(cwd.split('/')[0:-1]) + '/data/'\n",
    "#or you can hardcode the directory\n",
    "#datadir = \n",
    "\n",
    "\n",
    "\n",
    "#Load data and downsample for a 50/50 split, then split into a train/test\n",
    "f = datadir + 'ads_dataset_cut.txt'\n",
    "\n",
    "train_split = 0.8\n",
    "tdat = pd.read_csv(f, header = 0,sep = '\\t')\n",
    "moddat = bd.downSample(tdat, 'y_buy', 4)\n",
    "\n",
    "#We know the dataset is sorted so we can just split by index\n",
    "train = moddat[:int(np.floor(moddat.shape[0]*train_split))]\n",
    "test = moddat[int(np.floor(moddat.shape[0]*train_split)):]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Let's say we want to build a classifier that does ranking based on the probability of the label being 1. We can use Logistic Regression with AUC as our validation metric. We need to choose a regularization weight, but because we have limited data, we can do this via cross-validation.<br><br>\n",
    "\n",
    "The following function performs the cross-validation for various levels of C\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import *\n",
    "\n",
    "def xValAUC(tr, lab, k, cs):\n",
    "    '''\n",
    "    Perform k-fold cross validation on logistic regression, varies C,\n",
    "    returns a dictionary where key=c,value=[auc-c1, auc-c2, ...auc-ck].\n",
    "    '''\n",
    "    cv = KFold(n = tr.shape[0], n_folds = k)\n",
    "    aucs = {}\n",
    "\n",
    "    for train_index, test_index in cv:\n",
    "        tr_f = tr.iloc[train_index]\n",
    "        va_f = tr.iloc[test_index]\n",
    "    \n",
    "        for c in cs:\n",
    "            logreg = linear_model.LogisticRegression(C = c)\n",
    "            logreg.fit(tr_f.drop(lab, 1),tr_f[lab])\n",
    "            met = roc_auc_score(va_f[lab], logreg.predict_proba(va_f.drop(lab,1))[:,1])\n",
    "\n",
    "            if c in aucs:\n",
    "                aucs[c].append(met)\n",
    "            else:\n",
    "                aucs[c] = [met]\n",
    "    \n",
    "    return aucs\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now we can run our experiment</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "xval_dict = {'e':[], 'mu':[], 'sig':[]}\n",
    "k = 10\n",
    "auc_cv = xValAUC(train, 'y_buy', k, [10**i for i in range(-20,20)])\n",
    "for i in range(-20,20):\n",
    "    xval_dict['e'].append(i)\n",
    "    xval_dict['mu'].append(np.array(auc_cv[10**i]).mean())\n",
    "    xval_dict['sig'].append(np.sqrt(np.array(auc_cv[10**i]).var()))\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can now load the results from above into a dataframe and begin to analyze\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "res = pd.DataFrame(xval_dict)\n",
    "\n",
    "#Get the confidence intervals\n",
    "res['low'] = res['mu'] - 1.96*res['sig']/np.sqrt(10)\n",
    "res['up'] = res['mu'] + 1.96*res['sig']/np.sqrt(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now let's plot the results to get a sense of how AUC varies with regularization strength. We'll also plot the lower and upper 95% confidence bands to get a sense of the variance.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'X-validated AUC by C')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xl4lNXZ+PHvPTOZrARCCDskrGET\nQUABRah1oSrgLrgv1ValVt9WcasL1taqv4rWrWpxq4KKvkotovata0UBqQoEg2yBsGYlIdtMZu7f\nH89kHCCEgJkMJPfnuubKzDnPck8Ic89zznnOEVXFGGOMaYgr1gEYY4w59FmyMMYYs1+WLIwxxuyX\nJQtjjDH7ZcnCGGPMflmyMMYYs1+WLEyLIyLPi8jvQ8/HiUhuY7ZthrguE5HPmuNcjSUiKiJ9Yx2H\nOfRZsjBRJyIpIrJBRC6MKGsjIhtF5JxonltVP1XV7KY4loh8JCI/b4pjNXCOFBHZJSLv1lO31we7\niNwtIn+PeJ0qIrNCv9tdIrI29LpDNOMOnbuLiPxNRLaKSLmIfCci94hIcrTPbaLPkoWJOlXdBfwC\nmCUiGaHiB4ClqjovdpEdks4GaoCTRKTzgewoIl7g/4DBwEQgFRgDFAFHN3Gce567PbAISATGqGob\n4CSgHdAnmuc2zcOShWkWqvoe8E/gURGZAJwHXLuv7UVklYicHvHaIyIFInJU6PXrIrJNRHaKyCci\nMngfx5kgIvkRr4eLyLLQN99XgYSIujQReSd0npLQ8+6huvuAccBjoW/sj4XKB4jIByJSLCK5InJe\nxPHSRWS+iJSJyGIa96F5KfAU8C1wUSO2j3QJ0BM4U1VzVDWoqjtU9V5VXdDAfqeKyDoRKRSRB0XE\nJSLe0Hs6IuL9dBSRyoiEH+l/gHLgIlXdAKCqm1T116r67QG+D3MIsmRhmtONwARgHvBbVd3WwLZz\ngGkRr08BClV1Wej1u0A/oCOwDHh5fycPffN+C3gJaA+8jvNNvo4LeA7IxPnQrQIeA1DV24FPgemq\nmqKq00PNKx8Ar4TimAo8ISKDQsd7HKgGugBXhB4NxZeJ8/t5OfS4ZH/vaQ8nAgtDV3IH4kxgJHAU\nMAW4QlV9wFx2T1jTgP9T1YJ9nPtNVQ0e4LnNYcKShWk2qloCrASSgDf3s/krwGQRSQq9vgAngdQd\na7aqlqtqDXA3cKSItN3PMUcDccAsVfWHmsCWRByzSFXfUNVKVS0H7gPGN3C804ENqvqcqtaq6n+B\nN4BzRcSNk4juVNUKVV0BvLCf+C4GvlXVHJwP6sEiMnw/+0RKB7YewPZ1/qSqxaq6EZjFD0n6BWCa\niEhEfC818bnNYcKShWk2InIRkAX8C/hTRHnPUNPOLhHZBaCqa4BVwKRQwpiMk0AQEbeI3B/qvC0D\nNoQOtb9O3K7AZt199sy8iDiSROSvIpIXOu4nQLvQB399MoFjRKS07gFcCHQGMgAPsKm+c+3DJYSu\nkFR1M/AxTrNUnQBOsosUB/hDz4twrmIO1J4xdg3F8CVQCUwQkQFAX2D+Po5xsOc2hwlLFqZZiEhH\n4GHgKpzO7vNEZByAqm4MNe2kqGpKxG51TVFTgJxQAgHnKmMKTtNHW5wEBCA0bCvQLeKbMjjNTXV+\nA2QDx6hqKnD8Hsfdc4rmTcDHqtou4pGiqtcABUAt0GMf59qNiIzFaVa7NdQXsw04BrhARDyhzTZG\nvNc6vfghCf0LOOUgRh/tGeOWiNcv4DRFXQzMU9XqfRzjX8CZImKfKS2U/cOa5vIY8JaqfqiqW4Gb\ngWdEJL6BfeYCJwPXELqqCGmDM2KoCKdJ6w+NjGERzgf49SISJyJnsfsooTY4/RSlodE9d+2x/3ag\nd8Trd4D+InJx6HhxIjJKRAaqagCnqe3u0BXLIHa/StjTpTj9H4OAYaHHEJzRRT8LbfMqcIeIdA91\nQp8ITMLpAwKniWgT8Eao490V6mS/TURObeDcN4U693sAvw6dp87fcfo0LgJebOAYf8YZffVCqO8F\nEekmIn8WkaEN7GcOE5YsTNSJyBnAccBNdWWq+izON9g797VfKKksAsay+wfYizjfpjcDOcAXjYkj\n1Gl7FnAZUAycz+59J7NwPpwLQ8dcuMchHgHOCY2UejTUr3EyTsf2FmAbTvNaXQKcDqSEyp/H6Tzf\ni4gk4IwO+4uqbot4rMdJAHVJZibwOfAZUIIz/PjCUH8Iof6bE4HvcBJPGbAYp3nuywZ+NW8DXwFf\n44xY+1vE72wTzgACxengr5eqFuP8O/mBL0WkHGcY705gzb72M4cPscWPjDENEZHZwBZVvSPWsZjY\n8ex/E2NMayUiWThXYwcyKsu0QNYMZYypl4jcC6wAHgw1iZlWzJqhjDHG7JddWRhjjNmvFtNn0aFD\nB83Kyop1GMYYc1j56quvClW1vvm+dtNikkVWVhZLly6NdRjGGHNYEZH9zSwARLkZSkQmhmbiXCMi\nt9RTnyki/yci34qzVkD3iLpLReT70KOhm5mMMcZEWdSSRWg+ncdx7j4dhDMh2aA9NnsIeFFVh+Lc\ncPTH0L51d88eg3OH7V0ikhatWI0xxjQsmlcWRwNrVHVdxHTHU/bYZhDw79DzDyPqTwE+CM2EWYJz\nN+rEKMZqjDGmAdFMFt3YfTbL/FBZpG9wbvgBZ/6ZNiKS3sh9EZGrRWSpiCwtKKhvin1jjDFNIdZD\nZ38LjBeR/+KsG7AZZxrmRlHVp1V1pKqOzMjYb2e+McaYgxTN0VCb2X3q4+6hsjBV3ULoykJEUoCz\nVbVURDbjrBgWue9HUYzVGGNMA6J5ZbEE6CcivULLWU5lj4VTRKRDxPz3twKzQ8/fA04OTZuchjOz\n53tRjNUYY0wDonZloaq1IjId50PeDcxW1ZUiMhNYqqrzca4e/igiirMq2XWhfYtD89LULXk5MzQF\nsjGHrcrKSpYsWcLixYsJBAL89re/xePxsHLlSgoKCkhNTSU1NZU2bdrg8XhIT08HYOfOndTU1ISP\no6q4XC7qml4LCwupqqpCVVFVgsEgLpeLzMxMAHJzc9m5cye1tbXU1tYCkJqayrBhwwD47rvvqK6u\nxu1243K5CAQCJCQk0L9/fwA++eQTysrK8Pl8+P1+vF4vPXr0YOTIkeH93W43iYmJeL1e/H4/8fHx\ndOjgLFy4bNkyfD4fPp+PQCCAx+Oha9eu9OnTB1Vl5cqVeDwePB4Pbrcbt9tN27Ztadu2LcFgkNLS\n0nC5y+V8t4yLiyMuLo5gMEh1dXX49+L3+/H7/SQlJZGcnExNTQ3r1q3D7/eHzx0XF0eXLl1o27Yt\nfr+fnTt3EhcXh8vlwu/3U1NTQ7t27UhMTKS8vJw1a9aEY6+LoX///rRt25adO3eSl5eH2+0mISGB\nlJQUkpOTSUpKCsfakOLiYjZt2kRlZSXx8fEkJiaSmJhI9+7d8Xg8qCq7r9UVQ3V/YIf7Y8SIEWrM\noSIYDGowGFRV1VdffVVHjhypHo9HcdaFUBEJ1//85z8Pl9c92rRpEz7W+eefv1d9165dw/Wnnnrq\nXvUDBgwI148bN26v+pEjR4brhw0btlf9T37yk3B9nz599qqfMmVKuD4jI2Ov+ksuuSRcHx8fv1f9\nNddco6qqfr9/rzpAb775ZlVVLS4urrf+3nvvVVXVvLy8eusfeeQRVVVdvnx5vfV/+9vfVFV10aJF\n9dbPmzdPVVXfe++9eusXLlyoqqrz5s2rt/7TTz9VVdXXX39de/furUOGDNE+ffpoRkaGxsfH63ff\nfaeqqg899FC9+2/atElVVe+55x6Ni4vTlJQUTU5O1qSkJE1KStKysjJVVZ0xY4Z6vV6tqqpq/B/n\nHnC+vO/3M7bF3MFtzKHA7/dz22238corr/Dee+8xZMgQVJU2bdpw8803M3bsWEaPHk1SUlL4G+Mt\nt9zCBRdcQFlZGWVlZZSXl+N2/7Ds9xVXXMHxxx+/23lSUn5Yffb666/nzDPPREQQEVwuF+3btw/X\n//GPf6S0tJS4uDjcbjcistv+Dz/8MCUlJQSDwfC3706dOoXrX3/9dQKBAF6vF4/Hg8/nIzn5h5Vb\nn332WcrLy6mqqsLn8xEXFxe+KgF44403cLvdeL3e8JVL165dARARXn/9dWpra8Pf/gOBAEcccQQA\niYmJzJo1K1yuoYlPjzvuOADatWvHn/4UXs49fMUxbtw4AHr06MGcOXPC773uPMcccwwAmZmZPPro\no/j9foLBIF6vF6/XG77qGjZsGG+99VY49mAwSDAY5KijjgJg9OjRvPHGGwQCAaqrq9m1axcVFRX0\n6dMHgIyMDMaOHUtFRQVJSUnhq8fU1FQAJk2aRO/evUlMTKSmpoaqqiqqqqrCV5XHHnssv/nNb6ip\nqQn/+4oIcXHOUux17zPy7yVaWsyssyNHjlSb7sPEUnl5Oeeddx4LFy7krLPOYubMmQwePDjWYRnT\nIBH5SlVH7m87u7Iwpgls3bqV0047jW+//Za//vWvXH311bEOyZgmZcnCmCbw8ssvs3r1aubPn8+p\np54a63CMaXLWDGXMj1BdXU1CQgKqyrp168Jt1cYcLhrbDBXrO7iNOWzNnTuXfv36sXbtWkTEEoVp\n0SxZGHOAVJUHHniAadOm0bt3791GHhnTUlmyMOYA5Obmcu655zJjxgymTp3K+++/T1qazZ5vWj5L\nFsYcgCeffJKFCxfy+9//npdffpn4+PhYh2RMs7AObmMasG3bNu677z7OPvtsJkyYQHFxMbW1tXTs\n2DHWoRnTJKyD25gfobi4mFtuuYXevXvz5JNPsmzZMgDat29vicK0SnafhTF7ePrpp7npppsoLy/n\nggsu4O6776Zv376xDsuYmLJkYcweqqurGTNmDA899BBDhgyJdTjGHBKsz8K0erW1tfz5z3+mZ8+e\nTJ06lWAwGJ6wzZiWzvosjGmE5cuXM3r0aGbMmMG//vUvAFwulyUKY/ZgycK0Sj6fj7vvvpsRI0aw\nceNGXnvtNZ555pkmOfaqrWX8+7vtrNi8k4LyGoLBlnH1blo367MwrdKHH37IPffcw4UXXsisWbPC\nq7r9GPkllTywMJf532zZrdzjEjLaxNMxNYFObeLp3DaB7mmJZKYnk5meRGb7ZBK90V+PwJgfw5KF\naZVOOeUUlixZEl4a9Mcoq/bz+IdreO4/G3AJ/OqEvkzI7khBeQ07yqvZtrOa7WXO8w1FFSxaV0R5\nde1ux+iUGk9mejJZ6Un0SEsiIc6NyyW4BdwuCT13frpE8NUGqakN4KsNOo9AkJqI56pKIKgEghBU\nJRh6HQwt0+l1u/C4hDiPiziXEOd24XG7iHML1f4A5dW1lFX7Qz9rKQ89r6ypJdHroW2ih3ZJXtom\nxu32SIn3oCi1QSUYOn9AlUAwSCAIvtogFTW17PLVUlHjPHbVBNhV46eiJoDbJaQne0lL8tI+xUv7\nJC9pyV7Sk720S3IW/KnyBajyB6j0BagO/azyB6j2BQioouq8ZwVQUJRgEFwuSIzzkBzvJsnrIdnr\nJin+h59etwtQglr3O3Omdgmqs78vEHTO5aul0hc6ry9Apd/5mRDnol1SHGmh30u7JC/tEuNIS46j\nTUIc1f4AZVXO77Wsyh/66byuqAng9bhIjHOTEFf3002C102Cx4XX4yIQdH6vdT+D4ddBMtrEc+bw\n7j/6b7khlixMq/P1118zdOjQH50o/IEgcxZvZNa/vqe4wsdZR3XjplOy6dI2cb/77qz0k1dcwYai\nSvIKK8grriSvqIIPcwsoKK/Z7/57EoF4jwuv2/lgcYmTVJxEg5NoQskmqEptQPEHgvhDP2vrngeD\nJHjctEnwhB5xpCZ46J6WSGqChySvh0pfLTur/Oys8rOjvJrV28vZWeXfKwFGcoeSXZxbSI73kBLv\nITne+eDu1i6BlPgUkuM9BIJKUYWPkgofq7aWUVzho7TS3+D7Topzk+h1PlzdoWQqAAICzmuBoDqJ\nptJXS0VNAF8geMC/58j3U3fepNC5fbVBSip97Kzy09iWR5dAamIcyV4PNbVBavxO4qs9wKbLYT3a\nWbIwpilt2LCB4cOH8/DDD3PDDTcc1DFUlX+t2sEf313FuoIKRvduzx2nDWJIt7aNPkbbpDiGJrVj\naPd2e9VV+wP4A0GC4W/lP1wZBILON2evx+Ukh9DD44r96K1AUKn01f6QpMI/+VGx1QaClFb5Kanw\nIQKJXk/4gzrec/CDEXy1Qap8ASp8tVT6aqn2B0MJlVCydeKuex7ndpHkdc7rde/7vMGgUl5TS2ml\nk+hKq5wricQ4N6mJcaQmekhNiAslCXe9x/EHglT7A1T7nZ++QBCPy/l9elwuXC7wuFyh1055tEU1\nWYjIROARwA08q6r371HfE3gBaBfa5hZVXSAiWcAqIDe06Req+stoxmpah7lz5wIwZcqUg9r/q7xi\n/rQwl8Xri+mdkcyzl4zkpwM7NukHdUKoCeJw43YJbRLimvy4HreLDinxdEhp2nm46hJt26Smjdnl\nknCzXGb6wR0jzu0izu2iTUKThvajRC1ZiIgbeBw4CcgHlojIfFXNidjsDuA1VX1SRAYBC4CsUN1a\nVR0WrfhM6zRnzhzGjBlDr169Dmi/3G3lPPheLv9atZ0OKfHMnDKYaUf3JM5tAwpN6xDNK4ujgTWq\nug5AROYCU4DIZKFAauh5W2D3YSTGNKGcnBy+/fZbHn300Ubvk19SycMffM+b/80nxevhtyf35/Jj\ne5Ecby24pnWJ5l98N2BTxOt84Jg9trkbeF9EfgUkAydG1PUSkf8CZcAdqvrpnicQkauBqwF69uzZ\ndJGbFun111/H5XJx3nnn7Xfbol01PPbhGl7+YiMIXDWuN9eM70NasrcZIjXm0BPrr0fTgOdV9f+J\nyBjgJREZAmwFeqpqkYiMAN4SkcGqWha5s6o+DTwNznQfzR28ObzceuutnHTSSXTq1KnB7V5bsol7\n/rGSKn+A80b24Ncn9mvUCCdjWrJoJovNQI+I191DZZGuBCYCqOoiEUkAOqjqDqAmVP6ViKwF+gM2\n+ZM5aF6vl7Fjx+6z3h8Icu87Oby4KI+xfdKZOWUIfTumNGOExhy6otk7twToJyK9RMQLTAXm77HN\nRuCnACIyEEgACkQkI9RBjoj0BvoB66IYq2nhHnzwQX7/+9/vs75oVw0XPfslLy7K46pxvXjxiqMt\nURgTIWpXFqpaKyLTgfdwhsXOVtWVIjITWKqq84HfAM+IyI04nd2XqaqKyPHATBHxA0Hgl6paHK1Y\nTcsWCAR4+OGHOfroo+utX7F5J7946SsKd9Uw6/xhnDG8WzNHaMyhL6p9Fqq6AGc4bGTZnRHPc4Bj\n69nvDeCNaMZmWo9PPvmErVu3Mm3atL3q3v56MzPe+Jb2SV7m/XIsR3Rv/I11xrQmse7gNibq5syZ\nQ3JyMpMmTQqXBYLKAwu/46+frOPorPY8cdFRTX7TlzEtiSUL06L5fD7mzZvHlClTSEpKApzpNK5+\n6Ss+WV3AxaMz+d3pg/B67OY6Yxpi/0NMi1ZSUsIJJ5zAJZdcEi57b+U2PlldwN2TBnHvGUOaJFGU\nlpYye/ZsLr/8crKzszn++ON59dVX8fv3PQmeMYcTu7IwLVqnTp2YN2/ebmULlm+lY5t4LhmTdVDH\nDAQCfP3113z22WcMGjSIk046ieLiYq688krS09MZO3YsOTk5TJ06lX//+9/85Cc/aYJ3Ykxs2ZWF\nabGqqqpYu3btbmWVvlo+Xl3AxCGdcR3gTJ2qyvz58zniiCMYOXIkN9xwA++88w4AvXr1Iicnhx07\ndjB//nxWr17NBx98wIQJEwDnhsDLL7+cZcuWNcl7M6a5WbIwLdY//vEP+vbty+LFi8NlH+UWUO0P\nMnFI5wM+3oUXXsiUKVMIBAI899xzbNq0iUceeQRwprIeOHAgLpfzX8rlcnHiiSeGZ6NVVV5//XVG\njBjBcccdx8MPP8zy5cub4F0a0zxEtWXMkjFy5EhdutRu8DY/OOOMM1iyZAkbN27E7Xam/J7+yjIW\nrS3iy9t+iqcRM8bm5uaSmZlJQkICb7zxBoWFhVx55ZV4PAfegltaWspzzz3HU089xerVq5k+fTp/\n+ctfqK2t5Y477uCYY45hzJgxdO584Imsjt/vp6qqimAwSEpKCh6Ph5qaGsrKylBVVBWPx0NCQgKJ\niYnh5FbfcTweDyLCihUrWLRoEdu2baOwsJC0tDS6du3KRRddRFJSElVVVXi93vDveH+CwSAlJSWk\npzvzd7/00ku7/Tv179+fYcOGcf755zfqeKWlpezYsYOioiJUFZfLRadOncIzCy9fvhyXy0VcXBxp\naWm0b9++0bHuj9/vZ8uWLWzfvp2EhAS6dOlCenr6Pn+v9VFVfD4fXq83JmuSiMhXqrr/lcDq/oAO\n98eIESPUmDolJSXq9Xr1xhtvDJdV+Wp10O/e1Vve+Ga/+2/ZskWvvvpqdbvd+uc//7nJ49u8ebNu\n3LhRVVVzc3PV6/Uqzo2p2rNnTz3yyCP1nXfeUVXV5cuX68SJE3Xy5Ml67rnn6vnnn6+nnXaafvnl\nl6qqunDhQs3IyND4+PjwMQD96KOPVFX173//+27ldY+lS5eqqupzzz2nGRkZ2qNHD+3bt69mZGSo\niITju++++8L7tGnTJvy8oqJCVVVvvPFGdbvd2q1bNx02bJgOHz5cR48eHX6vt99+uw4dOlSHDh2q\nvXv31ri4OO3Ro0e4/vTTT9e2bdvqkCFDdMCAARoXF6djx44N10+YMEFHjRql06ZN06lTp+oJJ5yg\nt99+e7g+MTFxr/d21VVXqapqMBis973ffPPNqqpaVVWlp556ql588cV63XXX6fXXX6833HCDfvDB\nB6rq/B3ddtttescdd+gNN9ygU6dO1QkTJuhbb72lqqpffPHFXseOi4vT119/XVVVc3Jy9Je//KXe\neeed+tvf/lavuOIKnTJliq5cuVJVVV9++eXd/u2TkpI0PT1dv/nG+RudN2+ejh07VsePH69jxozR\no446SgcPHqz5+fmqqvrwww9rWlqa1tTUHPTfIs5N0vv9jLUObtMivfnmm/h8vt1uxPv0+0IqfAEm\nDunS4L6vvPIKV111FX6/n2uvvZYLL7ywyePr2rVr+Hn//v0pKytj2bJlfP755yxbtoyKigpSUpzp\nRnw+H8XFxdTU1ODz+QgEAqSkpFBZWQlAt27dOPvss0lNTaVNmzbhK4Y+ffoAMGrUKP7yl78g4qym\nFwgEqK6upkcPZ+q2Pn36cM4551BdXU11dTWpqal06dKFhARn5Z2rr76aiy66iE6dOhEfH4/f72f7\n9u3hocgTJ04kKSmJLVu2UFBQADjzcNXp0KEDvXr1QkRISEjgnHPO2W09kXnz5hEf/8M9LrW1tZSW\nloZfjxo1iq+//ppFixbh8XjIyMggLu6HBYseeeQREhMTSU9Px+12EwgE6N69+27HDwQC+Hw+SkpK\nKCoqYvTo0QBUVFSwbds2Vq5cGb76CgaDdOvWjRNPPJHS0lIefPBBgsEgiYmJdO7cmc6dOxMMBsP/\nds8++ywdO3akurqarVu3snXrVgYOHAjApk2bmDdvHoWFhSQmJtK+fXvat29PeXk5AAMHDuTGG2+k\nTZs2+Hw+KisrqaqqokOHDgB4PB4SExPx+/2kpKTQvn174uPjw1dGgwYN4sILL0SboYXImqFMizRx\n4kTWrFnD999/H760/5/Xvub/Vu1g6R0n7nPRohdffJHLLruMcePGMXv27PAHrjE/RiAQaLKmr6bW\n2GYou7IwLdIrr7zCunXrwonCVxvkg5ztnDyoc4Or2w0bNoxp06bx7LPPkpho05KbpnGoJooDYaOh\nTIvUvn17Ro784cvS52sLKa+u5Wf7GAX18ccfAzB06FBefvllSxTG7MGShWlxvvzyS/74xz+G24UB\nFq7YRkq8h+P6ddhtW1XlzjvvZMKECfzv//5vc4dqzGHDkoVpcRYuXMjtt98eHt5aGwjyfs52ThjQ\nkYS4H5oDVJWbb76Ze++9lyuuuILJkyfHKmRjDnmWLEyLU3dvRF1T0uL1xRRX+HZrggoGg0yfPp2H\nHnqI6dOn88wzz7SIdmVjosWShWlxcnNzyc7ODr9+d8U2EuJcjM/OCJctXryYp556iptuuolHH330\ngG6iMqY1stFQpkVRVVavXs24ceMACAaV91Zu4yfZHUny/vDnPnr0aL766iuOPPLImNw1a8zhxr5O\nmRZlx44dVFdXh68slm0sYUd5zW5zQW3fvh1VZdiwYZYojGkku7IwLUqnTp2orKwkEAgAsGD5Nrxu\nFycM6Ag4fRVHHXUUU6ZM4YknnohlqMYcVqJ6ZSEiE0UkV0TWiMgt9dT3FJEPReS/IvKtiJwaUXdr\naL9cETklmnGaliUuLo6EhARUnSao4/t3oE2CMz3EsmXL2LJlC2PGjIlxlMYcXqKWLETEDTwO/AwY\nBEwTkUF7bHYH8JqqDgemAk+E9h0Uej0YmAg8ETqeMQ168sknmTFjBgDf5u9kc2nVbnNBzZ8/H5fL\nxamnnrqvQxhj6hHNZqijgTWqug5AROYCU4CciG0USA09bwtsCT2fAsxV1RpgvYisCR1vURTjNS3A\n22+/HZ7M7t0V2/C4hJMGdtqt/rjjjgtPj22MaZxoNkN1AzZFvM4PlUW6G7hIRPKBBcCvDmBfRORq\nEVkqIkvrPiBM61Y3bFZVeXfFVsb27UDbJKcJasOGDXz77bd2850xByHWo6GmAc+ranfgVOAlEWl0\nTKr6tKqOVNWRGRkZ+9/BtGhVVVXk5eWRnZ3Nqq3l5BVV7nYjXseOHXnttdc477zzYhilMYenaDZD\nbQZ6RLzuHiqLdCVOnwSqukhEEoAOjdzXmN18//33qCrZ2dksXLEVl8DJg35ogkpKSuLcc8+NYYTG\nHL6ieWWxBOgnIr1ExIvTYT1/j202Aj8FEJGBQAJQENpuqojEi0gvoB+wGGMaUFJSQo8ePRgwYADv\nrtjGMb3SSU9xFtUpLS3l/vvvZ/Nm+85hzMGIWrJQ1VpgOvAesApn1NNKEZkpInWNxr8BrhKRb4A5\nwGWhlf5WAq/hdIYvBK5T1UA0jSQwAAAgAElEQVS0YjUtw/jx49m4cSM9+w3i+x27+MmAH5omFy5c\nyK233kpeXl4MIzTm8BXVm/JUdQFOx3Vk2Z0Rz3OAY/ex733AfdGMz7RMawt2AdC/U5tw2fz588nI\nyOCYY46JVVjGHNbsDm7TYpx//vkMGDCAAaddCUCfDGcNa7/fz4IFCzjrrLNsZlljDlKsR0MZ0yRU\nlXfffZeioiLW7thFQpyLbu2cKco//fRTdu7cyZQpU2IcpTGHL0sWpkXYtm0b5eXlZGdns6ZgF707\npOByOZMErlq1itTUVE488cQYR2nM4cuShWkRcnNzARgwYABrC3bRt2NKuO66665j+/btJCcnxyo8\nYw57lixMi1CXLDJ79yW/pCrcX6GqACQkJMQsNmNaAksWpkVITU1l/Pjx+OLTUCV8ZXH//fczbtw4\nfD5fjCM05vBmycK0CNOmTeOjjz5iXVElAH06Ok1Ob731Fn6/H6/XG8vwjDnsWbIwLUJdc9PaHbtw\nCWSlJ7NlyxYWL15sEwca0wT2mSxE5EER+UU95b8QkfujG5YxjVdTU0NaWhpPPPEEawp20aN9Eglx\nbt555x0ASxbGNIGGrixOAJ6up/wZ4PTohGPMgVuzZg07d+6kXbt2rN2xi76hzu23336bXr16MXjw\n4BhHaMzhr6E7uOO17to+gqoGxVa5N4eQ7777DoC+/fqzLqeA4/s7c0JNnjyZSZMmYX+uxvx4DSWL\nKhHpp6rfRxaKSD+gKrphGdN4dcNmUzJ64KvdHr6y+MUv9mpFNcYcpIaaoe4E3hWRy0TkiNDjcuCf\noTpjDgm5ubl07dqVbdXO6z4dk/nPf/5DYWFhbAMzpgXZ55WFqr4rImcAN/HDcqcrgbNVdXlzBGdM\nYxx33HH06dOHtTsqAOiVnkT2yDM4+eSTefnll2McnTEtQ4OzzqrqCuDSZorFmINy1VVXATBj3rd0\nSPFSVriNwsJCjj/++BhHZkzLsc9kISL/ACI7uBUoBD5U1b9HOzBjGsPn81FRUUFaWhprC3bROyOF\nnJwcABsFZUwTaujK4qF6ytoDF4nIEFW9JUoxGdNoX3zxBePHj+e9995jTQGcekQXVq36HICBAwfG\nODpjWo6G+iw+rq9cROYDXwGWLEzM1Y2E6tC1J6WVa+mTkcLn7+TQqVMn0tPTYxydMS3HAU/3YWth\nm0NJbm4u8fHxVMe3B5wJBG+//XZeffXVGEdmTMvSUJ9F+3qK04BLcEZF7ZeITAQeAdzAs6p6/x71\nDwM/Cb1MAjqqartQXQCoG3W1UVVtzgazl9zcXPr378+GYmfcbJ+MZLqnZdC7d+8YR2ZMy9JQn8VX\nOJ3adbe/KlAEfARcs78Di4gbeBw4CcgHlojIfFXNqdtGVW+M2P5XwPCIQ1Sp6rDGvQ3TWuXm5nLk\nkUeyZscuEuPcxNdW8tRTLzJp0iS6desW6/CMaTEa6rPota86EYlrxLGPBtao6rrQPnOBKUDOPraf\nBtzViOMaEzZjxgy6du3K3K276J2RzH//u4xrrrmGAQMGWLIwpgk1us9CHD8Vkb/hXCnsTzdgU8Tr\n/FBZfcfOBHoB/44oThCRpSLyRejmwPr2uzq0zdKCgoLGvRHTolx55ZX87Gc/Y80OZynVVatWATBo\n0KAYR2ZMy7LfZCEio0XkUSAPeBv4BBjQxHFMBebt0XmeqaojgQuAWSLSZ8+dVPVpVR2pqiMzMjKa\nOCRzqNuyZQvLly+nvLKGzaXOUqo5OTmkp6djfw/GNK2G1rP4g4h8D9wHfIvTn1Cgqi+oakkjjr0Z\n6BHxunuorD5TgTmRBaq6OfRzHU4/yfC9dzOt2UsvvcTQoUNZkbcdcEZC5eTkMGjQIJtp1pgm1tCV\nxc+B7cCTwEuqWsTud3TvzxKgn4j0EhEvTkKYv+dGIjIAZ5TVooiyNBGJDz3vABzLvvs6TCuVm5tL\n586d2VHjBqBPhtMMZU1QxjS9hkZDdcEZyTQNpxnoQyBRRDyqWru/A6tqrYhMB97DGTo7W1VXishM\nYKmq1iWOqcDcPdbOGAj8VUSCOAnt/shRVMaAs45Fdnb2D0updkhi7dq1VFdXxzo0Y1qchkZDBYCF\nwMLQt/zTgURgs4j8n6pesL+Dq+oCYMEeZXfu8fruevb7HDiiMW/AtF65ubmcc845rC2ooGf7JOI9\nbuLbtqVt27axDs2YFqdRo6FUtUZV31DVc4B+OEnEmJgpLCykuLiY7Ozs8EiohQsX8rvf/Y6amppY\nh2dMi3Mw032UqeqL0QjGmMZKTk7mnXfeYdLkKawvrKBPRgrvvPMOjz76KF6vN9bhGdPiHHCyMOZQ\nkJiYyGmnnUZcu874AsHwsNmBAwfaSChjoqDBxY+MOVR99tlnVFVV4eo+FIA+oWGzp556aowjM6Zl\namgiwbMa2lFV32z6cIxpnAcffJA1a9Zw/RPOoLp0t4/t27fbsFljoqShK4tJDdQpYMnCxExubi6D\nBg1ibcEuOqTEs7NoG23atLFkYUyUNDR09vLmDMSYxvL7/axdu5azzjqL73bsok9GMkceeSQ7d+4k\nGAzGOjxjWqRG9VmIyGnAYCChrkxVZ0YrKGMasn79empra+nfvz//zKvg9KFdABAR3G53jKMzpmVq\nzESCTwHnA7/CWdviXCAzynEZs091S6l26tGbnVV++mSk8Otf/5qZM+37izHR0pihs2NV9RKgRFXv\nAcYA/aMbljH7dvLJJ/Ptt9+S2MVZDa9vxxTefPNNvv/++xhHZkzL1ZhkURX6WSkiXQE/zrxRxsRE\nfHw8RxxxBPllzoz2HROC5OfnW+e2MVHUmD6Ld0SkHfAgsAxnJNQzUY3KmAY8/vjjZGZmslZ7keR1\nU7J5PWALHhkTTQ3dZxGnqn5VvTdU9IaIvAMkqOrO5gnPmL3NnDmTSZMmUTPmKnpnJPPdd7Y6njHR\n1lAz1GYReTa0lKpAeEJBSxQmZkpKStixYwfZ2dmsK6igb0YKbreboUOH0qvXPpeNN8b8SA0li4E4\nCxjdAWwSkUdEZHTzhGVM/epGQmX17hteSvWSSy7hm2++weOx2WuMiZZ9JgtVLVLVv6rqT4CjgXXA\nwyKyVkTua7YIjYlQlywSMpwVe/t2TIllOMa0Go1dz2IL8DecJVbLcZZcNabZ5eXl4fF48CV2AKBr\niosePXrw0ksvxTgyY1q2BpOFiCSIyLki8iawBjgBuAXo2hzBGbOnO++8k8LCQjaU1OB2CVUFG8nP\nzycpKSnWoRnTojU0GuoV4ETgY+Bl4AJVtcWNTcy1bduWtQVr6Nk+iTWrnWapgQMHxjgqY1q2hnoE\nFwK/UNXy5grGmIYEAgEuvvhiLrvsMtbuSKB3h2RWrVqFx+Ohb9++sQ7PmBatoQ7uF39sohCRiSKS\nKyJrROSWeuofFpGvQ4/VIlIaUXepiHwfelz6Y+IwLUNeXh5z5swhL28jecUV9OqQTE5ODv369bOl\nVI2JsqiNNRQRN/A4cBKQDywRkfmqmlO3jareGLH9r4DhoeftgbuAkTh3jH8V2rckWvGaQ1/dSKiO\nPXpRvbaazA7JuI8+muHDh8c4MmNavmgOTD8aWKOq6wBEZC4wBcjZx/bTcBIEwCnAB6paHNr3A2Ai\nMCeK8ZpDXF2y8KZ3B9aQlZ7ExbfeGtugjGklGruexVggK3J7VX1xP7t1AzZFvM4HjtnH8TOBXsC/\nG9i3Wz37XQ1cDdCzZ8/9hGMOd7m5uaSlpbFTnWVVuqR48Pl81gRlTDNozHoWLwEPAccBo0KPkU0c\nx1RgnqoGDmQnVX1aVUeq6siMjIwmDskcagKBACNGjCCvuIo4t7D4o4UkJSWxatWqWIdmTIvXmCuL\nkcAgVdUDPPZmoEfE6+6hsvpMBa7bY98Je+z70QGe37QwTz/9NADXvvwVPdKSWJ27GFUlKysrtoEZ\n0wo05g7uFUDngzj2EqCfiPQSES9OQpi/50YiMgBIAxZFFL8HnCwiaSKSBpwcKjOGDYWVZKYnkZOT\nQ+/evUlMTIx1SMa0eI1JFh2AHBF5T0Tm1z32t5Oq1gLTcT7kVwGvqepKEZkpIpMjNp0KzI28cgl1\nbN+Lk3CWADPrOrtN6/TNN98wduxYvvrqK/KKKshMd4bN2rTkxjSPxjRD3X2wB1fVBcCCPcru3ON1\nvcdX1dnA7IM9t2lZVqxYwaJFi6gKuqnwBejeNo7Vq1czadKkWIdmTKuw32Shqh83RyDGNCQ3NxeX\ny4UrtROwmW5tvdx3332MHTs21qEZ0yrsN1mE1rD4C876Fl7ADVSoamqUYzMmLDc3l169erFllzNg\nbkD3DH52000xjsqY1qMxfRaP4dww9z2QiDM9+ePRDMqYPeXm5pKdnU1eUQVul+Ar2UZ+fn6swzKm\n1WjsehZrALeqBlT1OZy7qY1pNgMGDGD8+PFsKKqkW7tE7rnrd4wbNy7WYRnTajSmg7syNPT1axF5\nANhKI5OMMU1l7ty5AEx+7DMy05NYsWqVTUtuTDNqzIf+xaHtpgMVODfanR3NoIyJVDeqWlVZX1hB\nZvsk1q5da9OSG9OM9pssVDUPEKCLqt6jqv8TapYyplk88cQT9OzZk7wtBZRX19Ihzkd5eTm9evWK\ndWjGtBqNmRtqEvA1zmJIiMiwxtyUZ0xTWbVqFTt37qS41mk1jassBLBkYUwzakwz1N04042XAqjq\n1zgzxBrTLOpGQm0srgJg1BHZvPLKK4wePTrGkRnTejSmg9uvqjtFJLLsQCcVNOag5ebmhkZCVSAC\nQ/v24OiBWbEOy5hWpTFXFitF5ALALSL9ROQvwOdRjssYACoqKti0aVPoHotKurZN5JtlS1m8eHGs\nQzOmVWlMsvgVMBiowVmprgy4IZpBGVOnurqaa665hnHjxrGhqILM9CTuvPNOrrvuuv3vbIxpMo0Z\nDVWpqrer6qjQQkO3q2p1cwRnTHp6Ok888QTjx48nr8iZmnz9+vXWuW1MM9tnn8X+Rjyp6uSG6o1p\nCqWlpaSkpFDhV4orfPRol0BeXh5nnnlmrEMzplVpqIN7DM462HOAL3HutTCmWV177bUsXbqUN/69\nBIA2wTJ8Ph+9e/eOcWTGtC4NJYvOwEk4kwheAPwTmKOqK5sjMGPAGQnVu3dvNhRVOAXlBYDdY2FM\nc9tnn0Vo0sCFqnopMBpYA3wkItObLTrTqqkqq1evDs82C3DK+DF8+umnHHPMMTGOzpjWpcH7LEQk\nHjgN5+oiC3gU+N/oh2UMbNmyhV27dpGdnc36oko6tomnY/t2dDzuuFiHZkyr01AH94vAEJxlUe9R\n1RXNFpUxOE1QANnZ2Xz0fSVZ6cn84x//QEQ4/fTTYxydMa1LQ0NnLwL6Ab8GPheRstCjXETKGnNw\nEZkoIrkiskZEbtnHNueJSI6IrBSRVyLKAyLydehhc1G1QllZWfzhD39g6NCh4XssHnjgAR566KFY\nh2ZMq7PPKwtV/VFrVoiIG2dFvZOAfGCJiMxX1ZyIbfoBtwLHqmqJiHSMOESVqg77MTGYw1vv3r25\n9dZbqfTVsqO8hqwOycxZv56TTjop1qEZ0+pEcxGjo4E1qrpOVX3AXGDKHttcBTyuqiUAqrojivGY\nw8w333zD9u3bySuqBKBLiovNmzfbSChjYiCayaIbzn0adfJDZZH6A/1F5D8i8oWIRC7XmiAiS0Pl\nZ9R3AhG5OrTN0oKCgqaN3sTcGWecwQ033BAeCRVXVQzYsFljYiHWy6N6cPpFJuCMuHpGRNqF6jJV\ndSTOPR6zRKTPnjur6tOhKUhGZmRkNFfMphlUVVWRl5dHdnY2G0JXFr6SrQB2Q54xMdCYKcoP1mac\nJVjrdA+VRcoHvlRVP7BeRFbjJI8lqroZQFXXichHwHBgbRTjNYeQNWvWoKpkZ2fzbVEF6clezjj9\nBDZs2ECnTp1iHZ4xrU40ryyWAP1EpJeIeIGpwJ6jmt7CuapARDrgNEutE5G00D0edeXHAjmYVqNu\n2OyAAQPYUOhMIOh2u8nMzCQhISHG0RnT+kQtWahqLTAdeA9YBbymqitFZKaI1E1C+B5QJCI5wIfA\nTapaBAwElorIN6Hy+yNHUZmW77vvvgOgf//+5BVVkJWezNNPP81zzz0X48iMaZ2i2QyFqi7Auakv\nsuzOiOcK/E/oEbnN58AR0YzNHNqmTp1K//79cXsT2LKzmsz0ZJ7601N07tyZyy+/PNbhGdPqRDVZ\nGHOw+vbtS9++ffl+ezkAWR2cdSzGjBkT48iMaZ1iPRrKmN2oKhdccAH//Oc/AcIjodLcfkpLS23Y\nrDExYsnCHFLefPNN5syZw44dzv2ZdfdYaNl2wO6xMCZWLFmYQ0ZtbS233XYbgwYN4pJLLgFgQ1EF\nbRPj2FVaiMvlsnssjIkR67Mwh4zZs2ezevVq3n77bdxuNwB5RZVkpSdx2mnHUVVVFS43xjQvu7Iw\nh4TKykruvvtujj32WCZNmhQud2abTQbA6/VasjAmRuzKwhwSEhISePDBB+nbty8iznLvvtogm0uq\nOHNYN+6++25SUlL47W9/G+NIjWmdLFmYQ4LL5eLCCy/crSy/pJKgQs/0ZO6YO5chQ4bEKDpjjDVD\nmZi79957+fOf/7xXed3U5D3TEtiwYYN1bhsTQ5YsTExt3LiR3//+9yxfvnyvug2hYbMJ/jJqamps\n2KwxMWTJwsTUXXfdhYhwzz337FWXV1RJstfNzh3OZMWWLIyJHUsWJmaWL1/OCy+8wPTp0+nZs+de\n9XUjoXbt2kWXLl2sGcqYGLIObhMzt912G6mpqdx666311ucVVTKwSxt+9rNxbNmypZmjM62d3+8n\nPz+f6urqWIfSJBISEujevTtxcXEHtb8lCxMzM2bMYNq0aaSnp+9VVxsIsqm4kolDOscgMmMgPz+f\nNm3akJWVFR7OfbhSVYqKisjPzz/o5lxrhjLNpqqqiscee4zzzz8fVeW4447jggsuqHfbLaXV1AaV\nrPQkfv7zn3P77bc3c7SmtauuriY9Pf2wTxQAIkJ6evqPukqyZGGirqysjD/96U9kZWXxq1/9is2b\nN7Nz584G96kbCZWZnsz777/Ppk2bmiNUY3bTEhJFnR/7XixZmKj68ssvyczM5JZbbmHYsGF8/PHH\nfPbZZ7Rr167B/fKKnXssuqR4yM/Pt85tY2LM+izMj1ZQUMDq1av5/vvvw48RI0YwY8YMhg4dyuTJ\nk5k+fTqjRo1q9DE3FFaQEOeiumQ7qmrDZo2JMUsWwKeffsof/vAHgsEgqkowGCQYDPL4448zcOBA\n/vnPf3L//ffjrAJL+OdLL71E7969ee2115g1a9Zex33zzTfp3Lkzzz//PE8//fRe9QsXLiQ1NZUn\nnniCv//973vVf/LJJ3g8Hh566CHefPPN3eq8Xi8fffQRADNnzmThwoW71bdr144FC5wVbW+55RY+\n+eST3eq7du3KvHnzAPj1r3/NkiVLdqvv27cvL774IgBXXHEFK1asCP9+VJUjjzwyvB72iBEjws1E\nbrebXr160b9/fwASExN54YUX9npvDdlVU8vbX29mRGYaeXkbALvHwphYi2qyEJGJwCOAG3hWVe+v\nZ5vzgLsBBb5R1QtC5ZcCd4Q2+72qHtgnzgGoqamhuLgYEcHlcuFyuRARgsEg4Mxb5PV66+IN/3S5\nnFa8uLg4UlJS9jpuXb3X6623vu5Y8fHx9dbXqa8+cvhbQkLCXvXJyckN1iclJYWfJyYmNlifmppK\nhw4dEJHw++7YsWO4/uGHHyYxMZF+/fqRlZV10EPz6vz147UU7vLx7CkDKMxdyjHHHEOfPn1+1DGN\n+THu+cdKcraUNekxB3VN5a5JgxvcZsOGDUycOJHRo0fz+eefM2rUKC6//HLuuusuduzYwcsvv8yC\nBQt2m2RzyJAhvPPOO2RlZTVpvFL3LbmpiYgbWA2cBOQDS4BpqpoTsU0/4DXgBFUtEZGOqrpDRNoD\nS4GROEnkK2CEqpbs63wjR47UpUuXRuW9mOazbWc1Ex76kJMGdeYv04bHOhzTiq1atYqBAwcCsU0W\nffv25b///S+DBw9m1KhRHHnkkfztb39j/vz5PPfccwwbNqzRySLyPdURka9UdeT+4o3mlcXRwBpV\nXRcKaC4wBciJ2OYq4PG6JKCqO0LlpwAfqGpxaN8PgInAnCjGaw4B/+/9XIJBuPmU7FiHYkzY/j7U\no6lXr14cccQRAAwePJif/vSniAhHHHEEGzZsYNiwYc0SRzRHQ3UDIsc75ofKIvUH+ovIf0Tki1Cz\nVWP3NS1MzpYy5i3L57Jjs+jR3mkGO/nkk5k+fXqMIzMmduLj48PPXS5X+LXL5aK2thaPxxNuMgei\ndsd5rIfOeoB+wARgGvCMiDQ8pjKCiFwtIktFZGlBQUGUQjTNQVX5w4JVpCbEcd2EvuHyZcuWUVtb\nG8PIjDm0ZWVlsWzZMsD5/7J+/fqonCeayWIz0CPidfdQWaR8YL6q+lV1PU4fR79G7ouqPq2qI1V1\nZEZGRpMGb5rXx6sL+GxNIdf/tB9tk5wO8rKyMoqKiuweC2MacPbZZ1NcXMzgwYN57LHHwiMRm1o0\n+yyWAP1EpBfOB/1UYM+5Hd7CuaJ4TkQ64DRLrQPWAn8QkbTQdicD9c82Zw57gaDyxwXfkZmexMWj\nM8Pldd+QbNisaa2ysrJYsWJF+PXzzz9fb937778f9ViilixUtVZEpgPv4Qydna2qK0VkJrBUVeeH\n6k4WkRwgANykqkUAInIvTsIBmFnX2W1annlfbSJ3ezmPX3AUXs8PF7t1ycKuLIyJvajeZ6GqC4AF\ne5TdGfFcgf8JPfbcdzYwO5rxmdirqKnl/72/muE923HqEbvPMJuens7ZZ59tycKYQ4DdwW1i6plP\n17GjvIYnLzpqr4nOxo0bx7hx42IUmTEmUqxHQ5lWbEdZNU9/so6fDenMiMz2e9X7fL4YRGWMqY8l\nCxMzD/9rNb7aIDMmDqi3fvjw4Vx66aXNHJUxpj6WLEyzU1U+/G4Hry7ZxMVjMsnqkFzvNuvXr6dD\nhw4xiNAYsyfrszDNpjYQ5J/Lt/LMp+tYsbmMbu0Suf6EfvVuu337dqqqqmzYrGnVUlJS2LVrV6zD\nACxZmGZQUVPL3CWbmP3ZejaXVtE7I5n7zzqCM4Z3IyHOXe8+69atA+weC2MOFZYsTNTsKKvm+c83\n8Pcv8iirrmVUVhp3Tx7MTwd0xOVqeIlHu8fCHGomTJiwV9l5553HtddeS2VlJaeeeupe9ZdddhmX\nXXYZhYWFnHPOObvV1a1H0xiqys0338y7776LiHDHHXdw/vnnc91113HKKacwefJkzjzzTNLS0pg9\nezazZ89m7dq13HfffQf6NvfJkoX5UXbV1JJXVMHGokryiivJK6pkY3EFeUWVbCmtQoGJgztz1fG9\nOapn2n6PV6d///7ceOONTT4nvzGHozfffJOvv/6ab775hsLCQkaNGsXxxx/PuHHj+PTTT5k8eTKb\nN29m69atgLOg29SpU5s0hlafLHZW+rnihSUEgs4KcAFVgkEIqoYeEAyGVsgL7RNeMS9GMTcXVVB+\n+H0Egs7vo+73FAgo5TW7T/KXlhRHz/RkjuqZxtlHdefM4d3q7cDen1GjRh3QMqzGRFtDVwJJSUkN\n1nfo0OGAriT29NlnnzFt2jTcbjedOnVi/PjxLFmyhHHjxjFr1ixycnIYNGgQJSUlbN26lUWLFvHo\no48e9Pnq0+qThbggMc6NCLhdgkvqHuASwe0SRCJWyKvbL/Sk4caUw58rtDqe27X3c5cInVITyExP\nomf7JHqmJ5Ga8ONWyQNn5syysrJ6L/uNMT/o1q0bpaWlLFy4kOOPP57i4mJee+01UlJSaNOmTZOe\nq9Uni9SEOP7+82NiHYYJCQaDXHvttWzcuJF169aRkJAQ65CMiblx48bx17/+lUsvvZTi4mI++eQT\nHnzwQQBGjx7NrFmz+Pe//01RURHnnHPOXv0jTaHVJwtzaHnllVf48ssvef755y1RGBNy5plnsmjR\nIo488khEhAceeIDOnZ251MaNG8f7779P3759yczMpLi4OCrT5ERtDe7mZmtwH/4qKirIzs6mc+fO\nLF68GJfL7hk1sVPfetWHu0N1DW5jDsiDDz7I5s2bmTt3riUKYw4x9j/SHDK6du3KL37xC4477rhY\nh2KM2YNdWZhDxtVXXx3rEIwx+2BXFibmFi9ezHPPPUcwGIx1KMaYfbArCxNTwWCQ66+/no0bN3Lu\nueeSkpIS65CMMfWwZGFias6cOeGhspYojDl0WTOUiZmKigpmzJjBiBEjuPjii2MdjjGmAVFNFiIy\nUURyRWSNiNxST/1lIlIgIl+HHj+PqAtElM+PZpwmNh566CE2b97MrFmzbKisMYe4qDVDiYgbeBw4\nCcgHlojIfFXN2WPTV1V1ej2HqFLVYdGKz8TeqFGjuPnmm22orDksxGKK8g0bNnD66aezYsUK/n97\ndx8cVXWHcfz7uAlEBRIQiGBAoqBitTZKQzWUCYoUsYov7QAdxJd2UFtJ1ekwdpxhnPqPIO0gUym1\nwpSio45VKWV8ATuCtVNfAQEJIkIqMAoYXyhStYFf/7gn4SZZ2JBscrfh95nZyd1zsnefnNzds/dl\nz4HoA9a+fftYuXIl5513HqtWraKuro6FCxdSXl7eqr+rpdrz41w5sMXMtprZ18DjwPh2fD6X48yM\nFStWcPPNN3Pw4EHGjRvHzJkzk47l3P+l/fv3s3btWubNm8dNN93U7s/Xnie4TwG2x+7vANKN2Het\npJHAZuAOM6t/TIGkN4E64D4zW9L0gZKmAlMBBg4cmM3sLou++OILFi9ezNy5c6murqZv375Mnz6d\n008/PelozrVYkkOUpzNp0iQARo4cyd69e/nss88oKirK6nPEJX2g+K/AIDP7JrACWBSrOzWMV/Ij\nYI6kZu8sZvaQmQ0zs7wkZjMAAAmDSURBVGF9+vTpmMTuqKxfv56SkhJuvfVWjj/+eBYtWsQHH3zg\nHYVzLZCXl9fo+0dffvllw3L9tAmHu5/1LO247p3AgNj9klDWwMxqY3cfBmbF6naGn1slrQTKgPfb\nK6zLzMw4cOAABw4cwMwaRoVdvnw5mzdvpqamhm3btlFTU8OIESN44IEHGDp0KBMmTOC6667joosu\navcN2rnOpLi4mN27d1NbW0u3bt1YtmwZY8eOBeCJJ55g1KhRvPLKKxQWFlJYWNiuWdpzz+INYIik\nUkldgIlAo6uaJPWL3b0SqA7lPSV1Dcu9gQqg6YnxrHn++ec588wzGTx4MKWlpQwcOJD+/fuzevVq\nAB555BF69uxJUVERhYWF9OjRgx49erBp0yYA5s2bR/fu3Zvdtm+PjqjNmjUrbf2nn34KwIwZM9LW\n19VFs9Ddeeedzep69+7dkH/q1KnN6uPTkU6aNKlZ/bnnnttQf8UVVzSrv/DCCxvqKyoqSKVSHHfc\nceTn51NQUMDo0aMb6qdNm8a0adN48MEH2bhxI3379qW0tBSIPhnNnz+fiooK7yicO0r5+fnMmDGD\n8vJyLr30Us4666yGuoKCAsrKyrjllltYsGBBu2dptz0LM6uTdBvwApACFprZO5J+BbxpZkuBKklX\nEp2X+AS4ITx8KPB7SQeJOrT70lxFlTVFRUWUlZWRSqVIpVLk5eWRSqUaeurBgwczZcoUINrVq7/V\nHx8855xz0o5rVP8ls7KysrT1Xbt2BWD48OFp6+vfXNO90eblHfrXVVZWNpsV68QTD01lOmbMGPr3\n79+ovlevXg3Ll19+OWeccUaj+n79DvXjkydP5uKLL27UPgMGHNppXLJkCT179qS4uNg7BOeyrKqq\niqqqqkZllZWVTJ48mTlz5nRYDp/Pwjnn0sjl+SwqKyuZPXs2w4ZlnIaiEZ/PwjnnjiHZvrKqJZK+\nGso553JWZznyAm3/W7yzcM65NAoKCqitre0UHYaZUVtb26Z57f0wlHPOpVFSUsKOHTvYs2dP0lGy\noqCggJKSklY/3jsL55xLIz8/v+EScOeHoZxzzrWAdxbOOecy8s7COedcRp3mS3mS9gD/asMqegMf\nZylOtnm21vFsrePZWuf/NdupZpZxJNZO01m0laQ3W/ItxiR4ttbxbK3j2Vqns2fzw1DOOecy8s7C\nOedcRt5ZHPJQ0gGOwLO1jmdrHc/WOp06m5+zcM45l5HvWTjnnMvIOwvnnHMZHdOdhaT7JW2StE7S\nM5KKYnW/lLRF0ruSvpdAth9KekfSQUnDYuWDJP1H0tpwm58r2UJdou3WJMs9knbG2mpcknlCprGh\nbbZIuivpPHGSaiStD22V+ExikhZK2i1pQ6ysl6QVkt4LP3vmULbEtzdJAyS9JGljeI3+PJS3vd3M\n7Ji9AWOAvLA8E5gZls8G3ga6AqXA+0Cqg7MNBc4EVgLDYuWDgA0Jt9vhsiXebk1y3gP8IuntLJYn\nFdrkNKBLaKuzk84Vy1cD9E46RyzPSOD8+PYOzALuCst31b9mcyRb4tsb0A84Pyx3BzaH12Wb2+2Y\n3rMws+VmVhfuvgrUj987HnjczL4ys23AFqC8g7NVm9m7HfmcLXWEbIm3W44rB7aY2VYz+xp4nKjN\nXBpm9jLwSZPi8cCisLwIuKpDQwWHyZY4M/vQzFaH5X8D1cApZKHdjunOoombgOfC8inA9ljdjlCW\nK0olrZG0StJ3kw4Tk4vtdls4zLgwqUMWMbnYPnEGLJf0lqSpSYc5jGIz+zAsfwQUJxkmjZzZ3iQN\nAsqA18hCu3X6+SwkvQicnKbqbjP7S/idu4E64NFcy5bGh8BAM6uVdAGwRNI3zGxvDmTrcEfKCfwO\nuJfoTfBe4NdEHwpceiPMbKekvsAKSZvCJ+icZGYmKZeu/c+Z7U1SN+Ap4HYz2yupoa617dbpOwsz\nG32kekk3AN8HLrFwQA/YCQyI/VpJKOvQbId5zFfAV2H5LUnvA2cAWT0h2ZpsdFC7xbU0p6Q/AMva\nM0sLdHj7HA0z2xl+7pb0DNFhs1zrLHZJ6mdmH0rqB+xOOlA9M9tVv5zk9iYpn6ijeNTMng7FbW63\nY/owlKSxwHTgSjPbH6taCkyU1FVSKTAEeD2JjE1J6iMpFZZPI8q2NdlUDXKq3cKLot7VwIbD/W4H\neQMYIqlUUhdgIlGbJU7SiZK61y8TXfyRdHulsxS4PixfD+TSXm7i25uiXYgFQLWZ/SZW1fZ2S/LM\nfdI3ohOw24G14TY/Vnc30ZUr7wKXJZDtaqJj2l8Bu4AXQvm1wDsh72rgilzJlgvt1iTnYmA9sC68\nWPrlwDY3jugKlfeJDuklmieW6zSiq7PeDttX4tmAx4gOu/43bG8/Bk4C/ga8B7wI9MqhbIlvb8AI\nosNg62Lva+Oy0W4+3IdzzrmMjunDUM4551rGOwvnnHMZeWfhnHMuI+8snHPOZeSdhXPOuYy8s3AO\nkLSvDY+9LYwia5J6x8olaW6oWyfp/KNcb5WkaknNRhaQVC7p5TCC7RpJD0s6obV/g3OZdPpvcDvX\nAf5B9G3dlU3KLyP6YuIQYDjRcBDDj2K9PwVGm9mOeKGkYuBJYKKZ/TOU/YBolNH9zdbiXBb4noVz\nMWFv4H5JG8LcDhNC+XGS5ima/2SFpGfDGzRmtsbMatKsbjzwJ4u8ChQ1+ZZv/XPeGZ5vg6TbQ9l8\noi/KPSfpjiYP+RmwqL6jCBn+bLHhJpzLNt+zcK6xa4BvAecBvYE3JL0MVBDNJXI20Jdo6OeFGdZ1\nuFFm60f/JAwGeSPRHoeA1yStMrNbwnA0o8zs4ybrPYdDw0071yF8z8K5xkYAj5nZgfBJfRXw7VD+\npJkdNLOPgJey+HzPmNkXZrYPeBrIpWHnnQO8s3CuPbXXKLPvABdkYT3OtZh3Fs419ndggqSUpD5E\n02e+TnQS+9pw7qIYqGzBupYCU8J5kO8An9uhCWjiz3eVpBPCaK9Xh7Ij+S1wvaSGk+WSrgm5nGsX\nfs7CucaeAS4kGn3VgOlm9pGkp4BLgI1E5yFWA59DdIkr0VD3JwPrJD1rZj8BniUa8XML0VVKNzZ9\nMjNbLemPHBrK/WEzW3OkgGa2S9JEYHaYqOgg0bwTz7flD3fuSHzUWedaSFI3M9sn6SSiN/eKcP7C\nuU7P9yyca7llkoqALsC93lG4Y4nvWTjnnMvIT3A755zLyDsL55xzGXln4ZxzLiPvLJxzzmXknYVz\nzrmM/gezJ546BQL/YAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "plt.plot(res['e'], res['mu'])\n",
    "plt.plot(res['e'], res['low'], 'k--')\n",
    "plt.plot(res['e'], res['up'], 'k--')\n",
    "\n",
    "plt.legend(loc = 4)\n",
    "ax.set_xlabel('log10 of C')\n",
    "ax.set_ylabel('Mean Val AUC')\n",
    "plt.title('X-validated AUC by C')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We see that AUC is definitely affected by the choice of regularization strength (here we seem to do better with less regularization, probably due to very strong signal-to-noise in the data). But its hard to see what is best in the range $1$ to $10^{30}$. We can zoom in to see this better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd8VFX6+PHPM5PeSKOXBKUGkBYQ\ncUVULGBddRFFFLEr4td19SerIoq9rGVtKyqosCgqa0WRRbquJiAtAaT3hBSSkDqZmfP7YyZjElIh\nk0nC83697isztz4zSe5zzz3nniPGGJRSSqnaWHwdgFJKqeZBE4ZSSqk60YShlFKqTjRhKKWUqhNN\nGEoppepEE4ZSSqk60YShWhwRmS0iT7pfnyUiW+uybiPENVFEVjXGsepKRIyIdPN1HKp50IShvE5E\nwkRkt4iMLzcvXET2isjV3jy2MWalMaZnQ+xLRJaJyC0Nsa8ajhEmIvki8l0Vy445uYvIdBGZU+59\nhIi84v5u80Vkh/t9rDfjdh+7vYi8JyKHROSoiGwRkcdFJNTbx1aNQxOG8jpjTD5wO/CKiLR2z34e\nSDbGfOa7yJqkq4AS4HwRaVefDUUkAFgC9AEuAiKAM4AsYGgDx1n52NHAz0AwcIYxJhw4H4gETvXm\nsVXj0YShGoUxZhHwLfCaiIwExgJ3Vbe+iGwWkUvKvfcTkQwRGeR+/6mIpIlIroisEJE+1exnpIjs\nL/d+oIisdV8BfwIElVsWJSLfuI9zxP26k3vZU8BZwOvuK/fX3fN7ichiEckWka0iMrbc/mJE5CsR\nyRORX6nbifNG4G1gA3B9HdYv7wagC/BnY0yqMcZpjDlsjJlhjFlYw3ZjRGSniGSKyAsiYhGRAPdn\n6lfu87QRkcJySb+8vwJHgeuNMbsBjDH7jDH3GmM21PNzqCZKE4ZqTPcBI4HPgL8ZY9JqWHcecG25\n9xcCmcaYte733wHdgTbAWmBubQd3X4F/AXwERAOf4rqiL2MBZgFxuE68RcDrAMaYh4GVwGRjTJgx\nZrL7Vsti4N/uOMYBb4pIgnt/bwDFQHtgknuqKb44XN/PXPd0Q22fqZJRwPfuEl19/BlIBAYBlwOT\njDE24GMqJq1rgSXGmIxqjr3AGOOs57FVM6IJQzUaY8wRIAUIARbUsvq/gctEJMT9/jpcSaRsX+8b\nY44aY0qA6UB/EWlVyz6HAf7AK8aYUvftsKRy+8wyxnxujCk0xhwFngLOrmF/lwC7jTGzjDF2Y8xv\nwOfAX0TEiisZTTPGFBhjNgEf1BLfBGCDMSYV18m6j4gMrGWb8mKAQ/VYv8xzxphsY8xe4BX+SNQf\nANeKiJSL76MGPrZqRjRhqEYjItcD8cB/gefKze/ivs2TLyL5AMaY7cBm4FJ30rgMVxJBRKwi8qy7\nQjcP2O3eVW0Vux2AA6Zij5t7ysURIiL/EpE97v2uACLdJ/+qxAGni0hO2QSMB9oBrQE/YF9Vx6rG\nDbhLSsaYA8ByXLeoyjhwJbzy/IFS9+ssXKWZ+qocYwd3DL8AhcBIEekFdAO+qmYfx3ts1YxowlCN\nQkTaAC8Dt+KqAB8rImcBGGP2um/zhBljwsptVnZb6nIg1Z1EwFXauBzXbZBWuJIQgFCzQ0DHclfM\n4Lr1VOZ+oCdwujEmAhhRab+Vu3beByw3xkSWm8KMMXcCGYAd6FzNsSoQkeG4brFNddfNpAGnA9eJ\niJ97tb3lPmuZrvyRiP4LXHgcrZIqx3iw3PsPcN2WmgB8ZowprmYf/wX+LCJ6TmnB9JerGsvrwBfG\nmKXGmEPAg8BMEQmsYZuPgQuAO3GXLtzCcbUkysJ1e+vpOsbwM66T+BQR8ReRK6nYeigcV71FjrvV\nz2OVtk8HTin3/hugh4hMcO/PX0SGiEhvY4wD12236e6SSwIVSwuV3YirPiQBGOCe+uJqdTTavc4n\nwCMi0sldMT0KuBRXnRC4bhftAz53V8Zb3BXvfxeRMTUc+wF3hX9n4F73ccrMwVXHcT3wYQ37+Aeu\nVlkfuOtiEJGOIvIPETmthu1UM6IJQ3mdiFwB/Al4oGyeMeZdXFey06rbzp1YfgaGU/Ek9iGuq+oD\nQCrwv7rE4a7IvRKYCGQD11CxLuUVXCfoTPc+v6+0i1eBq90tqF5z13NcgKuy+yCQhutWW1kSnAyE\nuefPxlWhfgwRCcLVauyfxpi0ctMuXEmgLNE8AfwErAKO4GqaPN5dP4K7PmcUsAVX8skDfsV1q+6X\nGr6aL4E1wDpcLdneK/ed7cPVqMDgqvSvkjEmG9fvqRT4RUSO4mrimwtsr2471byIDqCklKqJiLwP\nHDTGPOLrWJRv+dW+ilLqZCUi8bhKZfVpraVaKL0lpZSqkojMADYBL7hvj6mTnN6SUkopVSdawlBK\nKVUnLaYOIzY21sTHx/s6DKWUalbWrFmTaYypqn+wY7SYhBEfH09ycrKvw1BKqWZFRGrrgcBDb0kp\npZSqE00YSiml6kQThlJKqTrRhKGUUqpONGEopZSqE00YSiml6kQThlJKqTppMc9hqOOzdetWFixY\nQLt27SpMrVu3xs9P/zzKM8ZQUlKCMYbg4GBfh6NUo9MzwknGGMNvv/1GaWkpp59+OlFRUTzyyCM4\nnc4K67399tvcfvvtbN26lXvuuadCMmnbti3nnnsu7du35+DBg6xevdqz7zIjR46kTZs27Nmzh59/\n/hljDE6n0zNdcsklxMTEkJqayooVKyosczqd3HjjjURFRbF69Wq+++47SkpKKCkpwWazUVJSwquv\nvkpERAQfffQRH330kWcZgL+/P99//z0hISG8//77fP311/j7++Pn54e/vz/+/v7MnDkTEeGTTz5h\n6dKlFBYWeiZ/f3++/PJLAG6//Xb+85//eJYZY+jQoQMHDhwA4NZbbyU5OZmIiAhatWpFq1at6N69\nO9OmuYb5+Prrr8nPz8cY49lHmzZtGDduHADTpk1j7969FY4/ZMgQnnvONYLtlVdeSU5ODoGBgZ7p\nrLPOYvLkyQCe311gYCABAQH4+/szYMAARo0aBcCHH37o+cxl06mnnkrPnj2x2+1VPuzaqVMnOnXq\nhM1mY82aNcf8/cTFxdGxY0dycnL45ptvKC4upqioiKKiIoqLi7nkkksYNGgQO3bsYMaMGdjtdoKC\ngggKCiI4OJjx48czYMAADhw4wNdff+2ZHxwcTFBQEIMGDSI2NpYDBw7w888/U1hYSEFBgef7mTRp\nEh07dmT58uW8/fbbFBYWUlRUBICfnx9vvvkm8fHxfPfdd8yePRur1Yqfnx9WqxWr1cqzzz5LbGws\nK1asYOXKlcTGxhIbG0tMTAyxsbH06tWrzhdLdrudjIwM0tPTSUtLo6SkhMsvvxyA6dOns3r1agID\nA4mMjCQyMpJTTz2V++67D4AVK1Zgs9k8yyIjI2nVqhX+/pVH4a34/ZcNGFlaWorT6cQYQ2BgoGe+\nN2nCOElkZWUxd+5c3n//fdavX8+oUaNYvHgxbdq04ejRo54/+LS0NNLT0xkxwjU6aWFhIXl5eWzb\nto1Dhw5RUlICwFdffcWll17KmjVrGDt27DHHW7p0KW3atGH16tWMHz/+mOXJycnExMSwcuVK7rzz\nzmOWjxkzhqioKH755ReeeeYZz8kyICCAwMBACgsLiYiIoLi4mPz8fAICAggPDwdc/0gWi+tua3Z2\nNjt27KC0tBS73e75Jyv751q5ciVffPEFISEhnikmJsYTx8CBA7FYLJ5loaGhFZbHxcWRnp5OXl4e\n+/btY9OmTRw8+McIp48//vgxJ90zzzzTkzCWLl3K3r17Kxy/LHYAi8WCzWYjPz/fkzQ7duzoWf7O\nO+9w5MgR7Ha7Z97tt9/OqFGjPIm3svvvv58XX3yRwsJCzjjjjGOWT58+nccee4yMjAyGDx9+zPKX\nXnqJv/71r6SlpTFhwoRjlrdr145BgwZRUFDAsmXLsFqtFBcXexLLsGHDGDBgACkpKVX+7r/77jsu\nuugifvnlF/7yl78cs/ycc86hY8eOZGZmsmbNGkJDQwkKCgLA4XB4Ln4yMzNZv349DocDh8OB3W7H\n4XDw+OOPA7B8+XJPYi/vyJEjREZG8sQTT/Dhhx96EkpsbCyBgYH861//AuDmm29m1qxZFS6U2rZt\n60kYhw8fJj8/n+zsbDZv3kxOTg6nnHKKJ2Hcf//9xyTs4cOHey7A+vbty9atWz1JwRjDxRdfzDff\nfAO4/vYOHToEQFFRkec78KYW01ttYmKi0a5Bqvbwww/z4osvYrPZSExM5KabbuLaa68lKiqqXvsx\nxpCXl0daWhrt27cnIiKCo0ePsnfvXs86ZSfiuLg4QkNDyc3N9fxRW61WLBYLFouFDh06EBgYSEFB\nAXl5eRWWWSwWwsPDsVqtFa6omqPDhw+TnZ1dIemEhIQ0+D+3w+HAZrNRWlqK1WolNDQUYww7d+6k\ntLS0wtS2bVvi4+Ox2+3897//PWZf3bp1o1u3bhQXF7N8+fJjlvfq1Yu4uDhsNht79uw5poRgtVrr\nFLPNZiMrK+uYEkpCQgJRUVHk5OSwb98+QkNDK3x3DXmrtKSkhKysLDIzM8nMzCQjI4OxY8ciIsyb\nN4+vv/7aszwjIwM/Pz+2bduG1Wrl448/ZvPmzZ5Sd1kJ/JRTTqn2eOX/nrdu3crhw4fJyckhNzeX\nnJwcYmJiuPbaawH4xz/+QWZmpud/QkTo2bMn1113HQCvvfYa+fn5iAgPPPDAcX8vIrLGGJNYp3U1\nYbQ827dv54MPPuDBBx8kPDyc2bNns27dOiZNmsRpp+nwykqpP9QnYegtqRZk586d3HzzzSxbtgyL\nxcLw4cMZPXo0EydO9HVoSqkWQJvVthAOh4Prr7+e3377jWeeeYa9e/cyevRoX4ellGpBtITRQsyc\nOZOff/6ZOXPmVFnJrJRSJ0oTRgtxww034O/v76kQU0qphqYJo5mz2+2UlJQQGhrKzTff7OtwlFIt\nmNZhNHMvvfQS/fv3JzMz09ehKKVaOE0YzVhKSgrTpk1j4MCBFR4mU0opb9CE0UyVlpZy4403EhER\nwRtvvNGsH25TSjUPWofRTD3//POsWbOG+fPn06ZNG1+Ho5Q6CWgJoxlyOp0sWbKEa665psq+dpRS\nyhu0hNEMWSwWFi9e7OmhUymlGoNXSxgicpGIbBWR7SLyUBXLu4jIUhH5TUQ2iMgY93x/EflARDaK\nyGYRmerNOJuTzz77jPT0dKxWK2FhYb4ORyl1EvFawhARK/AGMBpIAK4VkYRKqz0CzDfGDATGAW+6\n5/8FCDTG9AMGA7eLSLy3Ym0u1q1bx7XXXstjjz3m61CUUichb5YwhgLbjTE7jTE24GPg8krrGCDC\n/boVcLDc/FAR8QOCARuQ58VYmzybzcbEiROJjY3l6aef9nU4SqmTkDfrMDoC+8q93w+cXmmd6cAP\nInIPEAqMcs//DFdyOQSEAPcZY7IrH0BEbgNuA+jSpUtDxt7kPPXUU6xfv54vv/yS6OhoX4ejlDoJ\n+bqV1LXAbGNMJ2AM8JGIWHCVThxAB6ArcL+IHDMqiTHmHWNMojEmsXXr1o0Zd6Nau3YtTz31FBMm\nTOCyyy7zdThKqZOUNxPGAaBzufed3PPKuxmYD2CM+RkIAmKB64DvjTGlxpjDwGqgTgN8tESdO3fm\nlltu4dVXX/V1KEqpk5g3E0YS0F1EuopIAK5K7a8qrbMXOA9ARHrjShgZ7vnnuueHAsOALV6Mtcky\nxtC6dWvefvvteg+pqpRSDclrCcMYYwcmA4uAzbhaQ6WIyBMiUnZf5X7gVhFZD8wDJhrXmLFvAGEi\nkoIr8cwyxmzwVqxNVVJSEiNGjKgwZrZSSvmKjundRGVkZDBixAjy8/PZuHEjkZGRvg5JKdUC6Zje\nzdz27dsZPXo0+/fvZ+HChZoslFJNgiaMJmb9+vWcf/75OJ1OfvzxR8444wxfh6SUUoDvm9WqSjp1\n6kRiYiI//fSTJgulVJOiCaOJ+PLLL7HZbMTExLBw4UJ69Ojh65CUUqoCTRg+Zozh4Ycf5oorruD1\n11/3dThKKVUtrcPwIZvNxq233sqHH37IrbfeypQpU3wdklJKVUtLGD6Sl5fHxRdfzIcffsgTTzzB\nv/71L/z8NH8rpZouPUP5yP79+/ntt994//33uemmm3wdjlJK1UoTRiNLS0ujbdu2JCQksHPnTiIi\nImrfSCmlmgC9JdWIVqxYQe/evXnjjTcANFkopZoVTRiNZMGCBZx//vm0bduWSy65xNfhKKVUvWnC\naAT//e9/GTduHIMHD+ann34iPj7e1yEppVS9aR2Gl2VlZXHVVVfRq1cv7RdKKdWsacLwspiYGGbN\nmsXpp5+uyUIp1azpLSkvOXz4MMuWLQPgyiuvpGPHjr4NSCmlTpCWMLwgPz+fMWPGsH37dnbv3q0l\nC6VUi6AJo4HZbDauuuoq1q1bxxdffKHJQinVYmjCaEBOp5NJkybxww8/8N5772nzWaVUi6J1GA3o\n008/Ze7cuTz55JNMmjTJ1+EopVSD0hJGAxo7dizBwcFceumlvg5FKaUanJYwGsAXX3zBjh07EBEu\nu+wyRMTXISmlVIPThHGCFi9ezNixY3nooYd8HYpSSnmVVxOGiFwkIltFZLuIHHNGFZEuIrJURH4T\nkQ0iMqbcstNE5GcRSRGRjSIS5M1Yj8fatWu58sor6dWrFzNnzvR1OBhjSMstptTh9HUoSqkWyGt1\nGCJiBd4Azgf2A0ki8pUxJrXcao8A840xb4lIArAQiBcRP2AOMMEYs15EYoBSb8V6PHbs2MHo0aOJ\njo7m+++/93nzWYfT8PcFG/kkeR/+VuGU2DB6tAunV7twerQNp2fbcDpFBWOx6O0ypdTx8Wal91Bg\nuzFmJ4CIfAxcDpRPGAYo6+O7FXDQ/foCYIMxZj2AMSbLi3Eel0cffRS73c6iRYvo0KGDT2Ox2Z3c\nN38d3244xIRhcYQF+fF72lF+23uEr9cf9KwXEmCle9twerYNo2e7CHq2DadHuzBahwVqvYtSqlbe\nTBgdgX3l3u8HTq+0znTgBxG5BwgFRrnn9wCMiCwCWgMfG2Oer3wAEbkNuA2gS5cuDRp8bWbOnMnO\nnTvp1atXox63suJSB3fNXcuPWw7z9zG9uG3EqRWWHy0uZdvhfH5PO8qWtKP8nn6UH7ccZn7yfs86\nrcMDGRIfxZD4aIbER9O7fQRWLYkopSrxdbPaa4HZxpiXROQM4CMR6euO60/AEKAQWCIia4wxS8pv\nbIx5B3gHIDEx0TRm4KGhofTr168xD3mM/BI7t36QzP92ZfHUn/sy/vS4Y9YJD/JnUJcoBnWJqjA/\nM7/Ek0Q27M8hafcRFm5MAyAs0I9BcVEMdSeR/p0jCfK3NspnaopKHU4y80s4nFdCel4xJXYnHSKD\naN8qmDbhgfhZfdN2xBhDXrGdzPwSsvJtlDqcdI4KoUNkkM9iUi2bNxPGAaBzufed3PPKuxm4CMAY\n87O7YjsWV2lkhTEmE0BEFgKDgCU0AWeddRaXXXYZDzzwgM9iyCm0MXFWEhsP5PLy2AFcMbB+nRvG\nhgUS2y2Q4d1iPfMO5BSRvDubX3dlk7Q7mxd/yAAgwGqhX6dWDImPZmjXKAbHRdMq2L9BPofd4aTA\n5qDQZqegxE5BicP10+agqNSBw+mk1GFwOA12p8HucOJwGve8P5aVOp04HAY/q4UgfwtB/laC/a2e\n157Jz0JwQNlrK35WISvfRnpeMelHi0nPK+FwXjGHj7qSQ3peCVkFJZhqLkesFqFteCDtI4Np3yqI\nDpHBdGgVRPvIYDq0CqZDZBDRoQEVbvkZ44rf7o6/1OHE7v5Z6nBidxpKSp1kF9rIPOo6fla+jQx3\nYsgqKCHzqOtnqePYwPwsQseoYLpEh9AlOoS4mBD361C6xIQQFljzv70xhkKbg5yiUnIKbeQWlrpf\nl5JXXIrd4cRpwGmM66fTeF4b43rtcLqWG2OwWITokACiQgOICf3jZ3RoAJEhAV4pzdodTk/8OYWl\nHCks5Yj7sxwptHGksBSb3Un7VkF0igqmY1QwHSOD6RAZfEIXRyV2B4fzSjiUW0xaXjGH84oJ9LMQ\nEezvmoL8aRXs53nd3C7EvJkwkoDuItIVV6IYB1xXaZ29wHnAbBHpDQQBGcAi4EERCQFswNnAy16M\ntc4yMzNZtWqVTx/OyzhawoT3fmFnRgFvjR/EBX3aNch+O0YG03FARy4f4Eo+OYU2kncfIWm3K4G8\nt2onby93naCC/V0nW3+rBT+L66e/VfCr4r2/VbCIUGRzkF9ip9BWlhTsFJeeeIsuq0WwWgQ/i2B3\nGmz249+nCMSEBtI2IpC2EUGc1qkVbcKDaBsRRJtw1zx/P+FQbjGHcoo5mFPEwdwiDuUUs+lALj+k\nph9z/EA/CwF+Fk9isDvrXxgO8LPQOiyQ2LAA2oQH0btdBLHhgcSEBriSf1ggFgvszy5iT3YBe7OL\n2JtVwLcbD5FTWLG9SExoAF1iQoiLDsFqsZBb5DqpliWF3CJblYmoOhYBiwgWi/zxWtyvLYLdYcgv\nsVf7fUcG+xPtTiCuKZBAPwsOp8FhDA6H66fTfdFQ/rXT/d7uMOQVu5JBTmEpR4urPh64/l6iQvzx\ns1g4fLSYyr+O2LBAOkYF0ynSlUg6uZNJx6hgBCEtr5j03GJPUkjPc71Ozysmu8BW5+8NXL/Xykkk\nNNDquZCw2V0XETaH8byuMN/uxOZw0q9jKz69Y3i9jn08vJYwjDF2EZmM6+RvBd43xqSIyBNAsjHm\nK+B+YKaI3IerAnyiMcYAR0TkH7iSjgEWGmO+9Vas9bFmzRoAhgwZ4pPjH8gp4vp3fyEtt5j3JiZy\nVvfWXjtWZEgAoxLaMiqhLQBFNgfr9uWwdu8RcotKK10Zu66Y7Q6DzeHE7r5SttmdFJe6SgUhAVY6\nh4YQGmAlNNDPNQX4ERpY/r3VMz/I3+JJQH5Wwc9S6bVVsLpPVOU5nIYSu4PiUidFpQ6KK0xOiksd\n7vmuf7qY0ABXQohwnXj963A7p1e7qsdjN8aQVWDjUE4xB3KKOJRbxCF3U2dPErW4fvpbXZ8vwJNY\n/1ge4GchOtSfmNBAYsMDCQ2w1q1hwqnHzsotKmVfdiF7sgrZk13geZ20+wjGGFqFBBAZ7E/3NmFE\nhvjTKjiAqBB/z+tI9+vI4AAigv3ws1iwupNDXRtLlNgd5BSWkpVvI7vARnahjez8ErILS8kuKOFI\nQSlZBSXszixkzZ4cbHaH+0LAgtUCfhYLFgtYRTwXCGXLyuZFhwZwauswWgX7ExXyR9xlr6NCAmgV\n4k94oJ8n7lKHk7Rc1+/qwJGiCj9TD+WxODUdWw3N1KNDA2gXEUT7VkEM6BxJ+1ZBtIsIom0r17w2\n4YHYHE7yiuzkFZeSV1RKblEpecV28opK3fP+eJ1TaONAjgN/99+Fv9X1txAS4Pr7CPT7428nwM/i\n+dk5KrhOv4cTJaa6snYzk5iYaJKTk71+nCeffJJHH32UnJwcWrVq5fXjlbcrs4Dr3/2FvKJSZt00\nhMT46EY9vlInG6fTkJlfwn53IjHgSRBtIgIJ9Gtet5Sq4q4fTqzLur6u9G52kpKS6NmzZ6Mniy1p\neVz/7q84jWHebcPo27Fxj6/UychiEdpEBNEmIuiYhiMnI00Y9TRo0CASE+uUjBvM+n053DjrVwL9\nLHx8yzC6tQlv1OMrpRRowqi3xx57rFGP97+dWdw8O4nosAD+fcswOkeHNOrxlVKqjDbWroejR49S\nWtp4PZQs3XqYG9//lfaRwXx6+3BNFkopn9KEUQ8vvPAC0dHRlJSUeP1YX/x2gFs/SKZbmzA+uW0Y\n7Vo1ub4XlVInGb0lVQ9JSUl07dqVwMBArx7n3ZU7efLbzZzeNZqZNyYSEdQwD8kppdSJ0BJGHRlj\nSEpK8urzF06n4emFm3ny282M6deODyYN1WShlGoytIRRR3v27CErK8trLaRKHU4e/GwD//ntABOG\nxTH9sj7aAaBSqknRhFFHSUlJgHee8C4osXPn3LWs+D2Dv13Qg7vP6abdjSulmhxNGHXUr18/ZsyY\n0eA91GbllzBptqsTweeu6sc1Qxq3m3allKor7RrEh/ZlF3LD+79yMKeI168bxPnuPpuUUqqx1Kdr\nkGorvUXkBRG5vYr5t4vIsycSYHPjdDr54YcfyMnJabB9phzM5cq3fiK7wMbcW07XZKGUavJqaiV1\nLu7BiSqZCVzinXCapu3bt3PhhReyYMGCBtnfTzsyueZf/8PPInx2xxnaiaBSqlmoqQ4j0FRxv8oY\n45SWViP7f/8H69ZVu7hVejpLgcS33oIPPzyhQ2UV2LAezmeuv4Ve7SIIXKQtm5VSJ2jAAHjlFa8f\npqazVZGIdK880z2vyHshNT1Hjx7FYrEQGnJiXXOk5RWzLf0oYYF+9OnQikA/TRZKqeajphLGNOA7\nEXkSWOOelwhMBf7P24E1qloy801nnYUzLo7Vy5cf9yHeWLqdFxZtZVTvtrx+3UD8mtnQjEopVe0l\nrjHmO+AK4Bxgtns6B7jKGLOwMYJrCux2O2vXrj2h5y9Wb8/kxR+2cln/Drx9/aBmN46vUkpBLc9h\nGGM2ATc2UixNksViYfXq1YSGhh7X9pn5JfzfJ+s4JTaUZ6/qh18dhv9USqmmqNqEISJf4xpPu4wB\nMoGlxpg53g6sqbBYLAwYMOC4tnU6DX/7dD25RaV8OGkoIQH6nKRSqvmq6Qz2YhXzooHrRaSvMeYh\nL8XUpHz66aeICFdffXW9t31/9S6Wbc3gicv70Lt9hBeiU0qpxlNtwjDGVFnDKyJf4aoEPykSxgsv\nvEBYWFi9E8bG/bk89/0Wzk9oy4RhcV6KTimlGk+9b6gbYxzeCKQpstlsrF+/vt491OaX2Lln3lpi\nwwJ5/qrTtCNBpVSLUFPXINFVTKeKyONASl12LiIXichWEdkuIseUSESki4gsFZHfRGSDiIypYnm+\niPyt3p+sAWzcuBGbzVbvFlLTvtzE3uxCXrlmAFGhAV6KTimlGldNdRhrcFV0l10eGyALWAbcWduO\nRcQKvAGcD+wHkkTkK2NMarlRCuJYAAAgAElEQVTVHgHmG2PeEpEEYCEQX275P4Dv6vRJvKCsM8P6\nlDD+89t+Fqw9wL3ndef0U2K8FZpSSjW6muowula3TETqMgzcUGC7MWane5uPgcuB8gnDAGW1wa2A\ng+WOcQWwCyiow7G8YsuWLcTExBAfH1+n9XdlFvDIfzYxND6ae87t5t3glFKqkdW5DkNczhOR93CV\nGGrTEdhX7v1+97zypuNqdbUfV+niHvexwoD/Bzxe1/i84eWXX2bbtm11qoOw2Z1MmfcbflYLr4wb\noM9bKKVanFrPaiIyTEReA/YAXwIrgF4NdPxrgdnGmE7AGOAjEbHgSiQvG2Pya4ntNhFJFpHkjIyM\nBgqpoqioqDqt9/z3W9h4IJfnrz6NDpHBXolFKaV8qaZK76dFZBvwFLABGAhkGGM+MMYcqcO+DwCd\ny73v5J5X3s3AfABjzM9AEBALnA48LyK7cfVb9XcRmVz5AMaYd4wxicaYxNatW9chpLr77bffuOaa\na9ixY0et6y7deph3V+1iwrA4LuzTrkHjUEqppqKmEsYtQDrwFvCRMSaLik9+1yYJ6C4iXUUkABgH\nfFVpnb3AeQAi0htXwsgwxpxljIk3xsQDrwBPG2Ner8exT9jKlSuZP38+QUFBNa53OK+Yv81fT692\n4Tx8ce9Gik4ppRpfTQmjPfAkcCmwQ0Q+AoJFpE79Wxhj7MBkYBGwGVdrqBQReUJELnOvdj9wq4is\nB+YBE6sag8MXkpOTad++PR07Vq52+YPTafjr/PUU2Oy8ft1A7VRQKdWi1dRKygF8D3wvIoG4RtkL\nBg6IyBJjzHW17dzdq+3CSvOmlXudCpxZyz6m13Ycb0hKSqq1Oe3bK3awansmz17Zj25twhspMqWU\n8o06NeUxxpQYYz43xlwNdMeVSFqsvLw8tm7dWuMDe2v3HuGlH37n4tPac82QztWup5RSLUW9u081\nxuQBJzZOaROXlpbGwIEDGTZsWJXL84pLmTLvN9pFBPH0n/tp1x9KqZOC9rddhR49erBmzZpql3++\nZj/7jxTx+Z1n0Cq4Ls8wKqVU86dPl1Whtnr3jQdyaR0eyOC46EaKqH6ys7N9HYJSqgWq6TmMK2ua\nGjPIxjZo0CAee+yxapenHsyjT4emMb7FunXruPPOO3nmmWcAWLBgAaeccgqffPKJjyNTSrU0NZUw\nLq1husT7oflGVlYW69atIyQkpMrlxaUOth3O92nCKCgo4P333+f0009n4MCBzJ4921OqGDhwIL17\n92bcuHHccsstFBT4rCsupVQLU1Oz2psaM5CmoqzuoroWUr+nH8XhNPTp0Koxw6rgjjvuYM6cOSQk\nJPDqq68yYcIETxcmXbt2ZcWKFUyfPp1nnnmG1atX8/HHH9O/f3+fxauUahnqVIchIheLyIMiMq1s\n8nZgvpKUlAS4bktVJeVgHgB9GylhFBYWMnv2bIYPH86WLVsAeOCBB1i5ciWbNm1iypQpx/R35e/v\nz1NPPcXixYvJzc1l/fr1jRKrUqplq7WVlIi8DYQA5wDvAlcDv3o5Lp9JSkqiR48eREZGVrl804Fc\nwoP86Bzt3Q4GS0pKePTRR5k5cyY5OTn07NmT9PR0evXqxWmnnVanfZx33nls2bKFiAjX7bNFixaR\nmJhITIyO06GUqr+6lDCGG2NuAI4YYx4HzgB6eDcs3zn77LO55ZZbql2ecjCPhPYRXn32wul0cvnl\nl/PCCy9w0UUXsWzZMjZv3szZZ59d732VJYu8vDzGjRtH//79WbZsWQNHrJQ6GdQlYRS5fxaKSAeg\nFFc/Uy3SfffdxwMPPFDlMofTsCUtz+v1FxaLhcsuu4z333+fefPmcfbZZ59wgoqIiGDJkiWEhIRw\n7rnnMm3aNOx2ewNFrJQ6GdQlYXwjIpHAC8BaYDfwb28G5SvZ2dkcPXq02uU7M/IpLnV6rYXUvn37\nWL58OQB33XUXN93UsO0OBg0axNq1a7nhhhuYMWMG55xzDjabrUGPoZRquaqtwxARf2NMqTFmhnvW\n5yLyDRBkjMltnPAa16uvvsrTTz9NXl4ewcHH1lGUVXj36djwCWPLli1ccMEFOBwOduzYUWu36scr\nLCyM2bNnc/7557NlyxYCAgJwOp3YbDavHVMp1TLUVMI4ICLvuodlFfB0QtgikwW4Krx79epVZbIA\nSDmYS4CfhVNbhzXocZOTkznrrLMoKSnhm2++aZQT9/jx45kxw3UtsGLFCtq3b8/dd99NcnJyrU+6\nK6VOTjUljN64BkF6BNgnIq+KSNW98bUAxhiSk5Nr7KE25WAevdqF49+A43UvWbKEc845h7CwMFat\nWsXAgQMbbN91FR0dzZgxY3j//fcZMmQI/fv35+WXX6aoqKj2jVWTYLPZyMnJ8XUYqoWr9sxnjMky\nxvzLGHMOMBTYCbwsIjtE5KlGi7CR7Nu3j4yMjGrHwDDGkOKFLkE+/fRT4uPjWb16Nd27d2/QfdfV\naaedxty5czl06BBvvfUWQUFBPPvss/j5ue5Ybtu27aStIDfGcODAARYvXsyrr77KHXfcwdSpUz3L\n09PTcTqdPolt1apV3HnnnQwZMoTw8HCioqI47bTTPE/9OxwOn8SlWq66jp53UETeA44Af8U1fOvD\n3gyssZU9sFddCeNAThG5RaUkNFALqfz8fMLCwnj99dfJz8+v9rmPxhQZGckdd9zBHXfcQWZmJv7+\n/jidTkaNGoXNZuOGG27gpptuolevXl6Lwel0kp2dTWRkJH5+fhw8eJDt27fjdDorTGeddRbBwcFs\n27aN1NRUz/zg4GAiIiIYOnQoAQEBFBcXY7Va8fevuVdhh8PB7t272bx5Mzk5OVx//fUAjBgxglWr\nVnnWi4qK4pprrvG8Hzx4MDk5OSQkJNC3b1/69u3LiBEjah18q67sdjupqakkJyezZs0akpOT+eij\nj+jRowcpKSnMmzePwYMHc++99xIREUFKSornQc6bbrqJdevWcc455zBy5EhGjBhRr2dwSkpK2LVr\nF9u3b/dM48eP54wzzmDbtm1MnToVPz8//P398fPzw8/Pj9tvv53ExES2bdvGzJkz8fPzIzQ0lLi4\nOOLj4+nfvz/h4Y0/2JjD4SAzM5O2bdsCMHXqVFJTU+nRowcJCQn06dOH3r17N2hshYWFHDhwgICA\nAKKioggLC8Niad79vdaYMEQkCFffUdcCw3ENnPQQsNj7oTWuwYMH889//pN+/fpVudxT4X2CJQxj\nDM8++yzvvvsuP//8M23atGkSyaKy2NhYwBXvq6++yqxZs3jppZd4/vnnGTZsGM8++yxnn302aWlp\nfPDBB4SGhlaYBg8eTLt27SgsLOTw4cP4+/uTmZlJeno66enpXHDBBbRt25Yff/yR5557jvT0dA4f\nPszhw4dxOBxs2rSJPn368PnnnzNlypRj4tu1axfx8fF8+umnPPzwsdcumZmZxMTEMGPGDJ5++mmC\ngoJo1aoVERERRERE8NNPPxEQEMC7777L66+/ztatWykuLgZcSWH8+PGICLfccgvXXnstCQkJ9O7d\nmzZt2niaOBtjmD59Ops2bSIlJYXvvvuOWbNmMWXKFBITEyktLeWCCy6gV69eREVFYYzBGMOYMWMY\nMWIEGRkZPPfcc559lU1XX301Z555JsuXL+eiiy7yxBUeHs6gQYPIz88HXAnh1ltvrfYkdMYZZ3Do\n0CFmzpzJa6+9BsCVV17J559/DrhOaAA7d+70JITExERGjhzJtm3b6NmzZ4X6rLJEfMYZZ1BYWMjm\nzZux2+0Vpssuc42+vHfvXl5//XVKS0srlE6///57LrzwQn788UeeffZZunbtSnx8vOdn//79q61D\nrI8lS5awYsUKNm/ezObNm/n999+Ji4vj999/B1x3FHbu3MmiRYsoKSkBXBcHZa0UX3jhBaKjoz2J\npFWriheKxhhycnLYs2ePZzrzzDMZPHgw69evZ9SoUWRmZlbY5oMPPuCGG25gzZo13HbbbURGRhIV\nFeX5OWnSJHr37u2Jq6CggIKCAvLz8ykoKODBBx8kPj6ehQsX8uyzz3rmFxQUsGvXrlovihpCTa2k\n/g2MApYDc4HrjDHFXo/IR+Lj45k8eXK1y1MO5mER6N3u+BOG0+nkb3/7Gy+//DLjx48/pkuPpshq\ntXLFFVdwxRVXkJaWxpw5c5g7dy5paWmA62Tz0EMPHbPd/Pnz+ctf/sLq1au54IILjlleduKw2+0c\nOXKEzp07k5iYSNu2bWnbti2tW7cG4IorriAhIQGr1YrFYvFM7dq1A1wnzdGjR2OxWBARiouLycvL\n8yTh888/n+DgYPLy8jzT0aNHPf9ce/bsoUOHDpx33nkkJCR4EkNZUrjxxhur/W7KEkp5mZmZnhNk\nVlYWdrudjz/+mPz8fEQEEaFNmzaMGDGC3Nxc3nrrLc+xypb37duXM888k549e3LnnXcyePBgEhMT\n6d69e4XkEBAQUOPv7s477+TOO+/EZrORlJTEsmXLPA9yOhwOOnfufExX+P/v//0/Ro4cSZcuXZg2\nbRrdunXzTDExMZ5Y+/fvT0pKSrXHPu+88zwJqaioiD179rB7925PCb64uJicnBz+85//kJGR4dku\nJSWFhIQE3nzzTZ577jlPycVqteLn58eyZcuIjo7mnXfe4aOPPvIsA9fAZxs2bMBisTBv3jxmzZrF\nKaecQq9evRg9ejR9+/b1HGfOnDme72HXrl2kpKQQGBgIuP5Pn3766Qp1Qp06deLuu+/moYceIi0t\njZ49e5KXl1fhMz/99NMMHjyY9u3bc+WVVxIXF0enTp08f+NDhw4FXM9ZtW/fniNHjrB582aOHDnC\nkSNHuPDCC+nduzfr1q3jrrvu8uw3KCiIsLAwJk6cSHx8PCKC1WqlY8eOngs0h8PRKAlDqmsRIyI3\nAP8xxlT/YEITkpiYaJKTk49rW6fTyeeff85ZZ53lORFVdvPsJPZmF7L4r/V/2hqgtLSUW265hQ8/\n/JApU6bw8ssvN/viKbiutAoLCz1XOmWvTz31VGJiYjhw4AA//PADJSUlxMbGehJCly5dtBmvDxUW\nFvLyyy9jjPEkhFNPPdUnFzH5+fns2bOHXbt2cf755xMYGMi3337LZ599VqH04nA4mD17NhEREbz3\n3nv8+9//9iwzxhAXF8c777xDeHg42dnZhISEHPffWNktypSUFFJTU0lNTeXcc89l4sSJ2O12/vrX\nvxIXF+e51RYXF0dsbOwJPWBrjEFEKCwsJC8vj9DQUEJCQjwJ0VtEZI0xpk73UKtNGM3NiSSMbdu2\n0aNHD959911uvvnmKtcZ9vQShp0SzSvjjq8V05QpU/jnP//JjBkzePjhh3VYV6VUk1CfhKFDtPJH\nhXd1FZVZ+SWk5RWfUJcgjz/+OOeeey5XXHHFce9DKaV8SRMGrgfngoKC6NOnT5XLT6TC22azYbFY\niIqK0mShlGrW6joexnARuU5Ebiib6rjdRSKyVUS2i8gxNaMi0kVElorIbyKyQUTGuOefLyJrRGSj\n++e59ftY9ZOUlMTAgQM9zx1UVpYwEo4jYTzzzDMMGTLE07JFKaWaq1oThoh8BLwI/AkY4p5qvd8l\nIlbgDWA0kABcKyIJlVZ7BJhvjBkIjAPedM/PBC41xvQDbgQ+qtOnOQ4Oh4O1a9fW8oR3Lh0jg4kM\nqblVSmXbt2/nmWeeoVevXoSFNWx3Ikop1djqcksqEUgw9a8dHwpsN8bsBBCRj4HLgdRy6xig7LK9\nFXAQwBjzW7l1UoBgEQk0xpTUM4ZaWSwWUlJSaqyETj2OJ7yNMdx9990EBgbyj3/840TDVEopn6tL\nwtgEtAMO1XPfHYF95d7vB06vtM504AcRuQcIxfXcR2VXAWurShYichtwG0CXLl3qGZ5nH8THx1e7\nvKDEzq6sAi4f0LFe+/3000/54YcfeO2112jfvsUOH6KUOonUpQ4jFkgVkUUi8lXZ1EDHvxaYbYzp\nBIwBPhIRT0wi0gd4Dri9qo2NMe8YYxKNMYllD3o1tM2H8jCm/hXe7777LoMGDarwAI5SSjVndSlh\nTD/OfR8AOpd738k9r7ybgYsAjDE/u7siiQUOi0gn4D/ADcaYHccZwwk73jEwvv32W9LT073+0I1S\nSjWWWhOGMWb5ce47CeguIl1xJYpxwHWV1tkLnAfMFpHeQBCQ4R7h71vgIWPM6uM8foNIOZhLdGgA\n7SLq9sTorl27iImJISIigk6dOnk5OqWUajx1aSU1TESSRCRfRGwi4hCRvNq2M8bYgcnAImAzrtZQ\nKSLyhIhc5l7tfuBWEVkPzAMmuivXJwPdgGkiss49tTnOz3hCNh1wVXjX5clsh8PBuHHjGDlypA5C\npJRqcepyS+p1XKWDT3G1mLoB6FGXnRtjFgILK82bVu51KnBmFds9CTxZl2N4k83uZNvho4zocUqd\n1p85cya//vorc+bM0a4/lFItTp0e3DPGbAesxhiHMWYW7nqHlu739KOUOkydKrzT09OZOnUq5557\nLtddV/nOm1JKNX91KWEUikgAsE5EnsfVvLb5d7NaB6n16BLkgQceoKCggDfeeENLF0qpFqkuJ/4J\n7vUmAwW4Wj5d5c2gmoqUg7mEBliJjwmtcT2bzcbhw4d58MEHvToanVJK+VJdWkntEZFgoL0x5vFG\niKnJSDmYR+/2EVgsNZcYAgIC+O6773QMZaVUi1aXVlKXAutwDc+KiAxowAf3miyn07D5UO1dgixY\nsIA9e/YgItV2XqiUUi1BXW5JTcfVL1QOgDFmHdDVizE1CbuzCiiwOWocA2PXrl1cf/31TJ06tREj\nU0op36hLwig1xuRWmtfiHzKorUtzYwxTpkzBYrHw3HPPNWZoSinlE3W5h5IiItcBVhHpDkwBfvJu\nWL6XcjAPf6vQo214lcu//PJLvvnmG1588UU6d+5c5TpKKdWS1KWEcQ/QByjB9TR2HvB/3gyqKUg5\nmEv3NuEE+B37FeXn5zNlyhT69evHlClTfBCdUko1vrq0kioEHnZPJwVjDKkH8zivd9W9kRw+fJjx\n48dz+eWX4+/v38jRKaWUb1SbMGprCWWMuaym5c1ZWl4xWQW2aiu8TznlFJ555plGjkoppXyrphLG\nGbgGQJoH/AKcNI8vpxyo+QnvTZs2ER8fr8OuKqVOKjXVYbQD/g70BV4FzgcyjTHLT6DL82Yh5WAe\nItC7/bEJw+l0MmzYMB5++KS5Q6eUUkANCcPd0eD3xpgbgWHAdmCZiExutOh8JOVgLl1jQgkNPLYA\ntmvXLgoKCjjttNN8EJlSSvlOjZXeIhIIXIxrKNV44DVco+C1aCkH8xjYJbLKZRs2bADQhKGUOunU\nVOn9Ia7bUQuBx40xmxotKh/KKbRxIKeI64fFVbl8/fr1iAh9+vRp5MiUUsq3aiphXI+rd9p7gSnl\nuuwWwBhj6jfIdTNRW5fmGzZsoHv37oSEhDRmWEop5XPVJgxjzEkx5kVlKbUkjKlTp5KRkdGYISml\nVJOg3atWknIwl3YRQcSEBVa5fMiQIY0ckVJKNQ0nZSmiJikHq+/SfO/evSxYsICjR482clRKKeV7\nmjDKKbI52JGRT5+OVT/hvWjRIq666ioyMzMbOTKllPI9vSVVzua0PJym5grv8PBw4uKqbkGllGoY\npaWl7N+/n+LiYl+H0mIEBQXRqVOnE+r/zqsJQ0QuwvWUuBV41xjzbKXlXYAPgEj3Og8ZYxa6l00F\nbgYcwBRjzCJvxgq1V3ivX7+efv36YbFowUwpb9q/fz/h4eHEx8dTroWmOk7GGLKysti/fz9dux7/\n+HdeO/OJiBV4AxgNJADXikhCpdUeAeYbYwYC44A33dsmuN/3AS4C3nTvz6tSD+bSKtifjpHBxywz\nxrBhwwZ9YE+pRlBcXExMTIwmiwYiIsTExJxwic2bl8pDge3GmJ3GGBvwMXB5pXUMUHY53wo46H59\nOfCxMabEGLMLV7ckQ70YK/BHhXdVf6T79u0jNzdXE4ZSjUSTRcNqiO/Tm7ekOuLq7bbMfuD0SutM\nB34QkXuAUGBUuW3/V2nbjpUPICK3AbcBdOnS5YSCLXU42ZJ2lBvPqLp+olOnTmzbto3IyKq7DFFK\nqZbO1zfjrwVmG2M6AWOAj0SkzjEZY94xxiQaYxJbt259QoHsyMjHZndWOwaGxWKhW7duxMbGntBx\nlFLNxxdffIGIsGXLlgbdb2ZmJv7+/rz99tsV5lceMmH27NlMnvxHf68ffvghffv2pV+/fgwcOJAX\nX3yxQeOqjTcTxgGg/GDXndzzyrsZmA9gjPkZCAJi67htg6ptDIx33nmHOXPmeDMEpVQTM2/ePP70\npz8xb968Bt3vp59+yrBhw+q13++++45XXnmFH374gY0bN/K///2PVq2qvsD1Fm/ekkoCuotIV1wn\n+3HAdZXW2QucB8wWkd64EkYG8BXwbxH5B9AB6A786sVYSTmYR5C/hVNaVz0o0iuvvEKPHj24/vrr\nvRmGUqqSx79O8fTx1lASOkTw2KU1dyCan5/PqlWrWLp0KZdeeimPP/44AOPGjWPChAlcfPHFAEyc\nOJFLLrmEMWPGMHHiRDZt2kTPnj05ePAgb7zxBomJicfse968ebz00ktcd9117N+/n06dOtUa8zPP\nPMOLL75Ihw4dAAgMDOTWW2+t70c/IV4rYRhj7MBkYBGwGVdrqBQReUJEyoZ3vR+4VUTW4xrZb6Jx\nScFV8kgFvgfuNsY4vBUruLoE6dUuAqvl2IqhoqIitm7dqhXeSp1EvvzySy666CJ69OhBTEwMa9as\nAeCaa65h/vz5ANhsNpYsWcLFF1/Mm2++SVRUFKmpqcyYMcOzfmX79u3j0KFDDB06lLFjx/LJJ5/U\nKZ5NmzYxePDghvlwx8mrz2G4n6lYWGnetHKvU4Ezq9n2KeApb8ZX7likHsrjsv4dqlyempqK0+nU\nhKGUD9RWEvCWefPmce+99wKuUsW8efMYPHgwo0eP5t5776WkpITvv/+eESNGEBwczKpVqzzr9+3b\nt9rzxSeffMLYsWM9+500aRL3339/tXE0pdZi+qQ3sC+7iKPFdvpW0yWIDpqk1MklOzubH3/8kY0b\nNyIiOBwORIQXXniBoKAgRo4cyaJFi/jkk08YN25cvfY9b9480tLSmDt3LgAHDx5k27ZtdO/eneDg\nYGw2GwEBAZ44yhra9OnThzVr1nDuuec27IetB1+3kmoSNh3MBaqv8N63bx+hoaGceuqpjRmWUspH\nPvvsMyZMmMCePXvYvXs3+/bto2vXrqxcuRJw3ZaaNWsWK1eu5KKLLgLgzDPP9NyqSk1NZePGjcfs\n9/fffyc/P58DBw6we/dudu/ezdSpUz2V32effbancU1RURHz58/nnHPOAVxDKzzwwAOkpaUBrtth\n7777rne/iEo0YeCqv7BahB5tw6tcPm3aNDIyMrBavf6wuVKqCZg3bx5//vOfK8y76qqrPCf2Cy64\ngOXLlzNq1ChPaeCuu+4iIyODhIQEHnnkEfr06XNMK6ba9vvqq6+yYMECBgwYwLBhw/jLX/7CiBEj\nABgzZgyTJ09m1KhR9OnTh0GDBpGX17CNAWojxphGPaC3JCYmmuTk5OPaduKsX0nLLeb7/xvRwFEp\npY7H5s2b6d27t6/DqBeHw0FpaSlBQUHs2LGDUaNGsXXrVk9CaQqq+l5FZI0x5timXFXQEgauJrUJ\n1dyOSktL4+KLL+ann35q5KiUUs1JYWEhf/rTn+jfvz9//vOfefPNN5tUsmgIJ32l9+GjxWQcLan2\nCe9169axcOFCHnzwwUaOTCnVnISHh3O8dzmai5M+YUQE+fPRzUPpGhta5fKyFlL9+vVrzLCUUqrJ\nOekTRpC/lbO6V98P1YYNG+jUqRPR0dGNGJVSSjU9WodRiw0bNtC/f39fh6GUUj6nCaMGxhhat27N\n8OHDfR2KUkr5nCaMGogIS5Ys4e9//7uvQ1FKNbLKXY03FJvNxk033US/fv3o378/y5Yt8ywbOXIk\nPXv2ZMCAAQwYMIDDhw8D8M9//pO+ffsyZswYbDYbAKtWreK+++7zSozV0YShlFKNaObMmQBs3LiR\nxYsXc//99+N0Oj3L586dy7p161i3bh1t2rTxzNuwYQPDhw9n0aJFGGOYMWMGjz76aKPGftJXetfk\n0UcfZfHixfz8889NqgMwpU42I0eOPGbe2LFjueuuuygsLGTMmDHHLJ84cSITJ04kMzOTq6++usKy\n8lf19bF7924mTZpEZmYmrVu3ZtasWXTs2JFu3bqxc+dOcnNziYmJYenSpYwYMYIRI0bw3nvv0b17\nd88+UlNTPf1BtWnThsjISJKTkxk6tPpRqI0xlJaWUlhYiL+/P3PmzGH06NGN3hhHSxg1SEpKwmaz\nabJQSgFwzz33cOONN7JhwwbGjx/PlClTsFqt9OzZk9TUVFatWsWgQYNYuXIlJSUl7Nu3r0KyAOjf\nvz9fffUVdrudXbt2sWbNGvbt+2M065tuuokBAwYwY8YMynrimDx5MsOGDWPv3r2ceeaZzJo1i7vv\nvrtRPzto1yA16tChAxdccAGzZ89u0P0qpWrWFLoGCQsLIz8/v8K82NhYDh06hL+/P6WlpbRv357M\nzEyeeuopoqOj2bVrF8OGDWPmzJk8/PDDvPbaa54OCcvY7XYeeOABli5dSlxcHKWlpdx22/9v786D\no6r2BI5/fyFoiiESFWVCCAiOCAGSsAph9Y2KUjpG0SHshnKZGhCVefWwhlIoBGoUEAcGB30OIpQC\njyhPFJTBICQSfKwxYZnngqBAMGETYpSX5Td/3JvQWemQdLqT/n2quuh77nZuurp/nHvu+Z0nSUxM\n5MSJE0RFRXHx4kVGjhzJuHHjmDBhQrn9Z8+eTWxsLCEhIaxcuZLo6GgWLlxISMiV//9vqUF8JC8v\nj5ycHEtpboy5oiFDhvjPaFcAABBaSURBVJCens6uXbsYMWIE58+fZ9u2bQwePLjStqGhoSxatIjM\nzEw+/PBDzp8/T+fOnQGIiooCnFHjY8aMYdeu8hONnjx5kl27dpGYmMjChQtZu3YtERERpKam+v4i\nsYBRrdLUxBYwjDGlEhISWLNmDeB0RJcGhH79+pGRkUFISAhhYWHEx8fzxhtvlGWa9VRQUMAvv/wC\nwJYtWwgNDSUmJoaioiJOnz4NQGFhIR9//DHdu3cvt+8LL7zA7NmzASf9uYgQEhJCQUGBz67Zk3V6\nVyM8PJxRo0ZZwDAmSBUUFJSba3vatGksWbKE5ORk5s+fX9bpDc782tHR0fTv3x+AwYMHs3r16ipT\nCuXm5jJ8+HBCQkKIiopi1apVAFy6dInhw4dTWFhIcXExd911V7k5u/fv3w9Ar169ABgzZgw9evQg\nOjq6wXLdWR+GMSbgBEIfRlNkfRg+cvbsWX9XwRhjAooFjCoUFRXRtm3bBh8UY4wxgcwCRhW++eYb\nLl26VOn5aWOMCWY+DRgicq+I/FVEvhWR56tYv0hEMt3X1yJy3mPdKyJyUEQOi8hiacDRc6VzYFiW\nWmOMucxnT0mJSDNgKXA3cBzYLSIbVPVQ6Taq+pzH9k8DPd33CcBAoPQRpS+AocA2X9XXU1ZWFqGh\noXTp0qUhTmeMMY2CL1sY/YBvVfWIqv4NWAM8WMP2o4HV7nsFwoBrgGuB5sBPPqxrOVlZWXTp0oVr\nr722oU5pjDEBz5cBIwr40WP5uFtWiYh0ADoCWwFUdSfwOZDjvjar6mEf1rWc5ORkpk+f3lCnM8YE\nmKNHj1YaNDdr1iwWLFjg9TEacxrz6gTKwL0kIEVViwFE5B+ArkDpqJktIjJYVdM9dxKRJ4EnAdq3\nb19vlXn44Yfr7VjGmODkmcY8NzeX++67j927d5flfHr33Xfp06f88IfSNObz5s1j8+bN3H///bz0\n0kusXr260vH9wZcB4wQQ7bHczi2rShLgmXrxIeBLVc0HEJFPgAFAuYChqm8Cb4IzcK8+Kn3q1Cly\ncnLo3r07zZs3r49DGmPqKFDSm3vWJy4uju3bt1NUVMTy5csrpSdvzGnMq+PLW1K7gdtEpKOIXIMT\nFDZU3EhEugDXAzs9in8AhopIqIg0x+nwbpBbUikpKfTq1ausiWiMMVUpKCggMzOT119/nUmTJlVa\n35jTmFfHZy0MVS0SkSnAZqAZsFxVD4rIbGCPqpYGjyRgjZbPUZIC/A7IxukA/1RVP/JVXT1lZWVx\nww030LZt24Y4nTHGCzW1CFq0aFHj+tatW9e6RVHdU/ye5aNHjwacTLUXLlzg/PnzRERElK2fNGkS\nhw8fpk+fPnTo0IGEhASaNWsGOLeePNOYr1q1igkTJjB+/HjGjx8POGnMp06dyieffFLrNOa+4tM+\nDFXdBGyqUPZiheVZVexXDDzly7pVJysri7i4OJs0yZggduONN3Lu3LlyZWfPnqVjx45lyxV/Iyou\nl6YxL5WQkFBjGnPPeS9K05i/+OKLDB06lK1btzJnzhxSU1O5++676+cir4KN9PZQUlJCdna2Zag1\nJsi1bNmSyMhItm7dCjjB4tNPP2XQoEFl26xduxZwnmJq1aoVrVq1KneMxpzGvDqB8pRUQDhy5AgF\nBQUWMIwxrFy5ksmTJzNt2jQAZs6cya233lq2PiwsjJ49e1JYWMjy5csr7d+Y05hXx9KbeygoKGDH\njh1069bN+jCM8aNAT28+bNgwFixYUOmx2EBX1/Tm1sLw0KJFC7/eHzTGmEBmAcPD+++/T+vWrRk6\ndKi/q2KMCWB1HcfRWFmnt4fp06ezdOlSf1fDGGMCkgUMV35+Pt999511eBtjTDUsYLgOHDgAYAHD\nGGOqYQHDVTppkgUMY4ypmgUMV3Z2NuHh4XTo0MHfVTHGmIBkAcP16quv8tVXX1lKEGNMlfNh1Na2\nbdvIyMiocZu0tDR69epFaGgoKSkptTr+ihUrmDJlSl2qWGv2WK2refPm5fLEGGMCxLPPQmZm/R4z\nPh5ee61+j1nBtm3baNmyJQkJCdVu0759e1asWFGriZn8yVoYOIm+pkyZwuHDDTapnzEmwBUVFTF2\n7Fi6du3KI488QkFBAXv37mXo0KH07t2b4cOHk5OTA8DixYuJiYkhNjaWpKQkjh49yrJly1i0aBHx\n8fGkp6dXeY5bbrmF2NjYShloc3JyGDJkCPHx8XTv3r1s/7fffpvOnTvTr18/duzYUbZ9Xl4eI0eO\npG/fvvTt27fcunqlqk3i1bt3b71aGzZsUEAzMjKu+hjGmPpz6NAhv57/+++/V0C/+OILVVVNTk7W\nV155RQcMGKC5ubmqqrpmzRpNTk5WVdXIyEj97bffVFX13Llzqqo6c+ZMnT9/vlfnmzhxoq5bt65s\necGCBTpnzhxVVS0qKtILFy7oyZMnNTo6WnNzc/XSpUuakJCgkydPVlXV0aNHa3p6uqqqHjt2TLt0\n6VLlear6u+JMN+HV76zdkuLyE1J1vWdpjGk6oqOjGThwIADjxo1j3rx5HDhwoCx9UHFxMZGRkYDz\ndOXYsWNJTEwkMTGxzufu27cvkyZNorCwkMTEROLj40lNTWXYsGHcdNNNAIwaNYqvv/4agM8++4xD\nhw6V7X/hwgXy8/Np2bJlneviyQIGTsDo1KkT4eHh/q6KMSZAVHwAJjw8nG7durFz585K227cuJG0\ntDQ++ugj5s6dS3Z2dp3OPWTIENLS0ti4cSOPPfYY06ZN47rrrqt2+5KSEr788kvCwsLqdN4rsT4M\nnIBh4y+MMZ5++OGHsuDw3nvv0b9/f/Ly8srKCgsLOXjwICUlJfz444/ceeedvPzyy/z888/k5+cT\nHh7OxYsXr+rcx44do02bNjzxxBM8/vjj7Nu3jzvuuIPt27dz5swZCgsLWbduXdn299xzD0uWLClb\nzqzvhwRcQR8wioqK+PXXXy1gGGPKuf3221m6dCldu3bl3LlzPP3006SkpDB9+nTi4uKIj48nIyOD\n4uJixo0bR48ePejZsydTp04lIiKCBx54gPXr19fY6b17927atWvHunXreOqpp+jWrRvgPGEVFxdH\nz549Wbt2Lc888wyRkZHMmjWLAQMGMHDgwHJpyhcvXsyePXuIjY0lJiaGZcuW+eRvYvNhuEpKSvw6\nV64x5rJAnw+jsarrfBj2C+myYGGMMTWzTm9jjPGxuXPnlutzAHj00UeZMWOGn2p0dSxgGGMCkqo2\nmVQ9M2bM8HtwqI/uB7sPY4wJOGFhYZw5c6ZefuSMEyzOnDlT58dufdrCEJF7gf8EmgFvqep/VFi/\nCLjTXWwB3KyqEe669sBbQDSgwAhVPerL+hpjAkO7du04fvw4eXl5/q5KkxEWFka7du3qdAyfBQwR\naQYsBe4GjgO7RWSDqpYNR1TV5zy2fxro6XGIlcBcVd0iIi2BEl/V1RgTWCwZaGDy5S2pfsC3qnpE\nVf8GrAEerGH70cBqABGJAUJVdQuAquaraoEP62qMMeYKfBkwooAfPZaPu2WViEgHoCOw1S3qDJwX\nkQ9EZL+IzHdbLBX3e1JE9ojIHmu6GmOMbwVKp3cSkKKqxe5yKDAY+D3QF+gEPFZxJ1V9U1X7qGqf\n0oRcxhhjfMOXnd4ncDqsS7Vzy6qSBEz2WD4OZKrqEQAR+TPQH/if6k62d+/e0yJyrA71bQ2crsP+\njZlde/AK5usP5muHy9fv9bzUvgwYu4HbRKQjTqBIAsZU3EhEugDXAzsr7BshIjepah7wO6DGvB+q\nWqcmhojs8XZ4fFNj1x6c1w7Bff3BfO1wddfvs1tSqloETAE2A4eBP6nqQRGZLSL/5LFpErBGPR64\ndm9N/R5IFZFsQIA/+qquxhhjrsyn4zBUdROwqULZixWWZ1Wz7xbAUsgaY0yACJRO70Dwpr8r4Ed2\n7cErmK8/mK8druL6m0x6c2OMMb5lLQxjjDFesYBhjDHGKxYwXCIyS0ROiEim+xrh7zo1BBG5V0T+\nKiLfisjz/q5PQxKRoyKS7X7eVz9dYyMhIstFJFdEDniU3SAiW0TkG/ff6/1ZR1+p5tqD4jsvItEi\n8rmIHBKRgyLyjFte68/eAkZ5i1Q13n1tuvLmjZtHgsj7gBhgtJvHK5jc6X7ewfA8/grg3gplzwOp\nqnobkOouN0UrqHztEBzf+SLg31Q1BmcA9GT3e17rz94CRnCrbYJI04ipahpwtkLxg8A77vt3gMQG\nrVQDqebag4Kq5qjqPvf9RZxxcVFcxWdvAaO8KSKS5TZfm2TTvAKvE0Q2UQr8r4jsFZEn/V0ZP2mj\nqjnu+1NAG39Wxg+C6jsvIrfgTCPxF67isw+qgCEin4nIgSpeDwL/DdwKxAM5wEK/VtY0hEGq2gvn\nltxkERni7wr5k5ttIZiesw+q77w7r9D7wLOqesFznbeffVDN6a2qd3mznYj8EfjYx9UJBLVJENnk\nqOoJ999cEVmPc4suzb+1anA/iUikquaISCSQ6+8KNRRV/an0fVP/zotIc5xg8a6qfuAW1/qzD6oW\nRk3cP1iph4AD1W3bhJQliBSRa3Dyem3wc50ahIj8nYiEl74H7iE4PvOKNgAT3fcTgQ/9WJcGFSzf\neRERnEzfh1X1VY9Vtf7sbaS3S0RW4TRNFTgKPOVxf6/Jch8lfA1n3vXlqjrXz1VqECLSCVjvLoYC\n7zX1axeR1cAwnLTWPwEzgT8DfwLaA8eAf1bVJtc5XM21DyMIvvMiMghIB7K5PNX1v+P0Y9Tqs7eA\nYYwxxit2S8oYY4xXLGAYY4zxigUMY4wxXrGAYYwxxisWMIwxxnjFAoYxgIjk12HfKW62XxWR1h7l\nIiKL3XVZItKrlsedKiKHReTdKtb1E5E0N9PwfhF5S0RaXO01GOONoBrpbYyP7MAZJbytQvl9wG3u\n6w6cVBR31OK4/wrcparHPQtFpA2wDkhS1Z1u2SNAOFBwFfU3xivWwjDGg9sqmO/mGMsWkVFueYiI\nvC4i/+fOHbDJ/ZFGVfer6tEqDvcgsFIdXwIRFUYXl55zmkdes2fdsmVAJ+ATEXmuwi6TgXdKg4Vb\nhxTPVBfG+IK1MIwp72Gc0b9xOKOCd4tIGjAQuAVn3pCbcVJEL7/CsarLBlw2mlhEegPJOC0PAf4i\nIttV9V9E5F6c+TpOVzhudy6npTamwVgLw5jyBgGrVbXY/R/7dqCvW75OVUtU9RTweT2eb72q/qKq\n+cAHwOB6OrYx9coChjG+46tswAeB3vVwHGNqxQKGMeWlA6NEpJmI3AQMAXbhdGyPdPsy2uAkrruS\nDcAEt1+kP/BzFcnt0oFEEWnhZs19yC2ryX8BE0WkrANdRB5262WMz1gfhjHlrQcGAF/hZDH9g6qe\nEpH3gX8EDuH0S+wDfgbn8VfgD8DfA1kisklVHwc2ASOAb3GeXkqueDJV3SciK3CCEsBbqrq/pgqq\n6k8ikgQsEJGbcTKQpgGf1uXCjbkSy1ZrjJdEpKWq5ovIjTg/8APd/gxjgoK1MIzx3sciEgFcA7xk\nwcIEG2thGGOM8Yp1ehtjjPGKBQxjjDFesYBhjDHGKxYwjDHGeMUChjHGGK/8PyzeS90VSrBEAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "\n",
    "res2 = res[(res['e']>-6)]\n",
    "\n",
    "best = res2['mu'].max()\n",
    "best_1sde = best - res2['sig'][(res2['mu']==best)]/np.sqrt(10)\n",
    "\n",
    "plt.plot(res2['e'], res2['mu'], label='Avg AUC')\n",
    "plt.plot(res2['e'], res2['low'], 'k--', label='Low 95%')\n",
    "plt.plot(res2['e'], res2['up'], 'k--', label='Up 95%')\n",
    "plt.plot(res2['e'], best_1sde.values[0]*np.ones(res2.shape[0]), 'r', label='best_1sde')\n",
    "\n",
    "plt.legend(loc=4)\n",
    "ax.set_xlabel('log10 of C')\n",
    "ax.set_ylabel('Mean Val AUC')\n",
    "plt.title('X-validated AUC by C')\n",
    "\n",
    "e_max = res2['e'][(res2['mu']==best)].values[0]\n",
    "e_1sde = res2['e'][(res2['mu']>=best_1sde.values[0])].values.min()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>So now we have 2 ways to select our regularization:\n",
    "<ul>\n",
    "    <li>Choose C with $max(AUC_{xval})$</li>\n",
    "    <li>Choose min(C) where $AUC_{xval}^C>max(AUC_{xval})-stderr(max(AUC_{xval})$</li>\n",
    "</ul><br>\n",
    "This latter decision criteria is more conservative and accounts for the fact that we have variance in our cross-validated estimates of AUC. We argue that any $C$ where $AUC_{xval}^C>max(AUC_{xval})-stderr(max(AUC_{xval})$ is statistically equivalent to the max. Therefore we take the most conservative, least complex model option.<br><br>\n",
    "\n",
    "Now that we have selected a model, let's retrain on the full training set and evaluate on the test. We'll bootstrap the testing estimation so we can get a sense of the variance.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def testAUCBoot(test, nruns, model, lab):\n",
    "    '''\n",
    "    Samples with replacement, runs multiple eval attempts\n",
    "    returns all bootstrapped results\n",
    "    '''\n",
    "    auc_res = []; oops = 0\n",
    "    for i in range(nruns):\n",
    "        test_samp = test.iloc[np.random.randint(0, len(test), size=len(test))]\n",
    "        try:\n",
    "            auc_res.append(roc_auc_score(test_samp[lab], model.predict_proba(test_samp.drop(lab,1))[:,1]))\n",
    "        except:\n",
    "            oops += 1\n",
    "    return auc_res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lab = 'y_buy'\n",
    "logreg_max = linear_model.LogisticRegression(C = 10.0**(e_max))\n",
    "logreg_max.fit(train.drop(lab,1), train[lab])\n",
    "auc_max = testAUCBoot(test, 100, logreg_max, lab)\n",
    "\n",
    "logreg_1sde = linear_model.LogisticRegression(C = 10.0**(e_1sde))\n",
    "logreg_1sde.fit(train.drop(lab,1), train[lab])\n",
    "auc_1sde = testAUCBoot(test, 1000, logreg_1sde, lab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now let's look at the distribution of AUC across the bootstrapped samples. Even though we can't use the test data for model selection, we can at least look at the test results for models built with the 2 selection criteria discussed above.\n",
    "\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'Distribution of BootStrap Test AUC')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt8FdW5//HPE4IEyx2CIiEEDioI\n4i3VoyAWqdR6RX9oUYtc5XhetSjaKlovsbUtp6c9tNZa6/GCFRV71Cpqa4tUilCUgiIXQUFNMYAa\nQBAFlMDz+2MWcROSsCH7kmS+79drvzJ7ZvZaz6y9M8+eNbPXmLsjIiLxlZPtAEREJLuUCEREYk6J\nQEQk5pQIRERiTolARCTmlAhERGJOiaCRMLN7zOyWFJVVaGafmlmT8HyWmY1NRdmhvD+b2YhUlbcf\n9d5hZuvN7INM1y1SnykRNABmVmpm28xsi5ltMrN/mNmVZlb5/rn7le7+oyTL+npt67j7andv4e47\nUxB7iZlNrVL+N939obqWvZ9xFALXAUe5+6HVLP+ame0KCfBTM1tjZrenqO692sDMepvZX81sY3hP\nF5rZWQmxlKWi7iTjuydhu78wsx0Jz/9ch3KvNLMXk1x3Wqi7QzXzb64yr6eZVVSZN8LMXjOzz8xs\nnZk9Z2YnH2jscaNE0HCc6+4tga7AJOAG4P5UV2Jmuakus54oBDa4+0e1rLM2JMAWQH9gjJkNSVM8\nzwIzgEOBjsB44JNkX5zK9yl8idi93T8BHt/93N2/map6amJmrYHzgS3AJQfw+puI/idKgHyi/5H7\ngPNSF2Uj5+561PMHUAp8vcq8E4FdQJ/wfApwR5juADwHbAI2Ai8TJf2Hw2u2AZ8C1wNFgANjgNXA\n7IR5uaG8WcBPgflEO6tngHZh2deAsuriBc4EvgB2hPreSChvbJjOAW4G/gV8BPweaB2W7Y5jRIht\nPfCDWtqpdXh9eSjv5lD+18M27wpxTKnmtdVtxx+AmxKenwL8E9gc/p6SsOwwYHpo71XAFWH+Xm0Q\n3h8H2lQTx1eqxPppKLsEeAKYGt6DseEzMC+8z+uAu4CDEspyogTzbmi7/wZy9vFZKwGmVjP/VODV\nUNdrQL+EZVeE93xLqOsi4DhgO1ARtuGDWuocF9rsemBBlWXTgJurzOsJVCR81rcTfVHK+v9qQ31k\nPQA9kniTqkkEYf5q4D/D9BS+TAQ/Be4BmobHqYBVVxZf7mx/H3ZCzak+EawB+oR1nty9s6CWRBCm\n99qxsGciGB12At2BFsBTwMNVYvvfENcxwOdArxra6fdESapleO3bwJia4qzy2j2WA4eHbT49PG8H\nfAwMB3KJvrl+DLQPy2cDdwN5wLFEyej06toAMGAlUbIeAhxSWywJZewI6+eE9jgB+PcQTxGwHLgm\n4TUOvBRiLwztMXYfn7Xq3q8iYANRQs0Bzgrb1zY8NgH/FtY9bPf7A1wJvJjE53su8EOgC1EC7J2w\nbF+JYAhR4rRs/5825Ie6hhq2tUT/5FXtADoBXd19h7u/7OG/phYl7v6Zu2+rYfnD7r7U3T8DbgEu\n3n0yuY4uA/7H3d9190+BG4FhVbo+bnf3be7+BtE36mOqFhJiGQbc6O5b3L0U+AXRjjtZh4X++k+I\ndpqvAnPCsrOBle7+sLtXuPtjwArgXDPrAvQDbnD37e6+iKhr4vLqKgnvxUCihPkLYJ2ZzTazw/cR\n3zx3f9rdd4X2WOjur4R4SoHfAadVec1/uftGd18N/JID6HohOiJ7yt1fDHX/CXgTGJywTh8zy3P3\nte6+PNmCwzafAjzq7u8THb1W2241aA98mMTnW2qhRNCwdSbqiqjqv4m+Zf/VzN41s4lJlPX+fiz/\nF9GRRoca1t0fh4XyEsvOBQ5JmJd4lc9WoiOHqjqEmKqW1Xk/Ylnr7m3cvRXQhuib5u6T2lXjTCz/\nMGCju29Jtm53L3P3q9z934j6tD8jOqKpzR7vkZkdEU6KfhCS10/Y+z2p+r4dto86qtMV+HZIkpvM\nbBNQDBzm7h8TJfPxwAdmNt3MeuxH2ZcDr7n7ivD8kVDX7i8ZFUTva6KmRF92IDpSOcTMbP83S3ZT\nImigzOyrRDuaOVWXhW/E17l7d6ITZtea2aDdi2socl/fqLokTBcS/SOuJ9qBHZwQVxOiE3bJlruW\naEeTWHYF8OE+XlfV+hBT1bLW7Gc5ALj7ZuBR4Nwa4kwsfy3Qzsxa1lB3rW0Qvgn/hqjrrbb1q87/\nLdFRyeEhed1E1O2UqOr7tra2WGrwPnBfSJK7H19x98kh/ufdfRBRklkd4qptOwAIO+/hQK+QzD4g\nSmaHEXVDEcorqvLSbnyZlOcQbfPZB7BdEigRNDBm1srMziHqO53q7kuqWeccM+sR/tE2AzuJ+l4h\n2sF2P4Cqv21mR5nZwUT9uU94dHnp20CemZ1tZk2JTtA2S3jdh0BR4qWuVTwGTDCzbmaWeNVKRQ3r\nVyvE8gfgx2bW0sy6AtcSnVzdbyGWYcCyMOtPwBFmdqmZ5ZrZt4CjgOfCjvwfwE/NLM/M+hKdfN9d\n9x5tYGZtzez28B7lhEsmRwOvJKzfPlxNU5uWRCeOPzWznsB/VrPO90N9XYCrgcf3uzGio6KLzGyQ\nmTUxs+Zh+lAz6xze+4OJzt98yp6ftS7hc1GdrxF1YR5PdF7lWKJk+CRfdg/9H3ChmZ0e2qoLUcKb\nBuDu64EfAb8Ln/vmZtbUzM41s58cwLbGU7ZPUuix7wdRX/I2oqsyNhNdKfIdoEnCOlP48mTxhPCa\nz4Ay4JaE9c4n+pa1CfgeVU4Mh3X2mMfeVw09C3RIWH8k0VUrH4UyS/nyZHF7om9tHxN1AewuL/Gq\noVuJvnWWE+0821YXR9XXVtNObcPry0N5txKukiG5k8WJV+psAJ4HeiSs0x9YGN6DhUD/hGUFRCd/\nNwLvAFcmLNujDYhOuD8U2ulToq6vx4DOCa95IMSwiS+vGqp6EncA0RHBp0R96z8E5iQsT7xqaAPR\n+YgmNbVBeM1e9YT5/RK24SOiK6QOIzrKeDl8LjYBM4mOUCA6cf6X8Jq92p7oM/tINfMHEHUBtgrP\nLwQWhTpKiT6Lzaq8ZiTwOtFnfh3RZ/SkbP/vNpTH7itJRKSRMTMn2imvynYsUr+pa0hEJOaUCERE\nYk5dQyIiMacjAhGRmGsQA4x16NDBi4qKsh2GiEiDsnDhwvXunr+v9RpEIigqKmLBggXZDkNEpEEx\ns6q/hq+WuoZERGJOiUBEJOaUCEREYq5BnCMQkfpvx44dlJWVsX379myHEjt5eXkUFBTQtGlNwzrV\nTolARFKirKyMli1bUlRUhEaFzhx3Z8OGDZSVldGtW7cDKkNdQyKSEtu3b6d9+/ZKAhlmZrRv375O\nR2JKBCKSMkoC2VHXdlciEBGJOZ0jEJG06Dfpb6zZVNMtsPdf5zbNmTvx9JSVJ19SIhCpRqp3YulQ\n33eMazZto3RS6u4gWTTx+ZSVtb+efvppFi9ezK233srnn3/O5ZdfzsKFC2nfvj2PP/441Q2BM3ny\nZO677z7MjKOPPpoHH3yQvLw8xowZw4IFC3B3jjjiCKZMmUKLFi2YMmUK3//+9+ncObrV9VVXXcXY\nsWMpLy9n+PDhvPDCC2nbPiUCkWqkeieWDtncMcbNz372M6ZPnw7A/fffT9u2bVm1ahXTpk3jhhtu\n4PHH97wD6Jo1a7jzzjt58803ad68ORdffDHTpk1j5MiRTJ48mVatWgFw7bXXctdddzFx4kQAvvWt\nb3HXXXftUVZ+fj6dOnVi7ty59OvXLy3bp3MEItJolJaW0rNnT0aOHMkRRxzBZZddxosvvki/fv04\n/PDDmT9/PvPnz+fkk0/muOOO45RTTuGtt94Com/wo0ePBmDJkiX06dOHrVu38vbbb9OsWTM6dOgA\nwDPPPMOIESMAGDp0KDNnzqS64fwrKirYtm0bFRUVbN26lcMOOwygMgm4O9u2bUvqRO+QIUN45JFH\n6t5ANVAiEJFGZdWqVVx33XWsWLGCFStW8OijjzJnzhx+/vOf85Of/ISePXvy8ssv8/rrr/PDH/6Q\nm266CYCrr76aVatW8cc//pFRo0bxu9/9joMPPpi5c+dy/PHHV5a/Zs0aunTpAkBubi6tW7dmw4YN\ne8TQuXNnvve971FYWEinTp1o3bo1gwcPrlw+atQoDj30UFasWMF3v/vdyvlPPvkkffv2ZejQobz/\n/vuV84uLi3n55ZfT0l6gRCAijUy3bt04+uijycnJoXfv3gwaNKiyn760tJTNmzdz0UUX0adPHyZM\nmMCyZcsAyMnJYcqUKQwfPpzTTjutshtm3bp15OfvcyTnPXz88cc888wzvPfee6xdu5bPPvuMqVOn\nVi5/8MEHWbt2Lb169arsVjr33HMpLS1l8eLFnHHGGZVHHQAdO3Zk7dq1dW2aGikRiEij0qxZs8rp\nnJycyuc5OTlUVFRwyy23MHDgQJYuXcqzzz67xw+xVq5cSYsWLfbY6TZv3nyPdTp37lz5bb2iooLN\nmzfTvn37PWJ48cUX6datG/n5+TRt2pQLL7yQf/zjH3us06RJE4YNG8aTTz4JQPv27StjHTt2LAsX\nLqxcd/v27TRv3rxO7VIbnSwWkbTo3KZ5Sk9od26Tmh3h5s2bK6/MmTJlyh7zx48fz+zZs7nqqqt4\n4oknGDp0KL169drj2/x5553HQw89xMknn8wTTzzB6aefvlc/f2FhIa+88gpbt26lefPmzJw5k+Li\nYtydd955hx49euDuTJ8+nZ49ewLRkUenTp0AmD59Or169aos7+2336ZPnz4p2f7qKBGISFrU10tb\nr7/+ekaMGMEdd9zB2Wd/eWXYhAkT+M53vsMRRxzB/fffz8CBAxkwYAADBgzguuuuw90xM8aMGcPw\n4cPp0aMH7dq1Y9q0aQCsXbuWsWPH8qc//YmTTjqJoUOHcvzxx5Obm8txxx3HuHHjcHdGjBjBJ598\ngrtzzDHH8Nvf/haAO++8k+nTp5Obm0u7du32SFIvvfTSHrGmnLun5QE8AHwELE2Y99/ACmAx8Eeg\nTTJlnXDCCS6SSV1veC7bIexTfYvxzTffzHYIaTN+/HifMWNG1uo/9dRTfePGjbWuU137Aws8iX1s\nOs8RTAHOrDJvBtDH3fsCbwM3prF+EZGUuOmmm9i6dWtW6i4vL+faa6+lbdu2aasjbYnA3WcDG6vM\n+6u7V4SnrwAF6apfRCRVDjnkEM4777ys1J2fn8+QIUPSWkc2rxoaDfy5poVmNs7MFpjZgvLy8gyG\nJSISL1lJBGb2A6ACqPGncu5+r7sXu3vx/l7DKyIiycv4VUNmNhI4BxgUTmaIiEgWZTQRmNmZwPXA\nae6enTMvIpIZk4+GzatTV17rQpiwJHXlSaW0JQIzewz4GtDBzMqA24iuEmoGzAg/wHjF3a9MVwwi\nkkWbV0PJ5tSVV9J6n6uMHj2a5557jo4dO7J06dKkiy4qKmLBggWVA8tVx90ZNGgQTz/9NK1ateKF\nF17g6quvZufOnYwdO7ZyBNFEq1evZsSIEWzatImdO3cyadIkzjrrLObPn8+4ceMqyy0pKeGCCy5g\n+/btDBgwgM8//5yKigqGDh3K7bffDsCwYcP40Y9+xOGHH570diUtmWtMs/3Q7wgk0+rbNfrVqW8x\n7nUd+22tUltBEuX9/e9/94ULF3rv3r33q+iuXbt6eXl5res899xzfs0117i7e0VFhXfv3t3feecd\n//zzz71v376+bNmyvV5zxRVX+N133+3u7suWLfOuXbu6u/tnn33mO3bscHf3tWvXen5+vu/YscN3\n7drlW7ZscXf3L774wk888USfN2+eu7vPmjXLx44dW2N89fV3BCIiGTVgwADatWu3x7w777yTo446\nir59+zJs2DAANmzYwODBg+nduzdjx47dYxjpqVOncuKJJ3LsscfyH//xH+zcuROARx55hPPPPx+A\n+fPn06NHD7p3785BBx3EsGHDeOaZZ/aKx8z45JNPgGgIi91DUR988MHk5kYdMtu3b68cosLMaNGi\nBQA7duxgx44dlctOPfVUXnzxRSoqKqpWU2dKBCLSqE2aNInXX3+dxYsXc8899wBw++23079/f5Yt\nW8YFF1zA6tXRuYzly5fz+OOPM3fuXBYtWkSTJk0q7wMwd+5cTjjhBGDPoagBCgoKWLNmzV51l5SU\nMHXqVAoKCjjrrLP49a9/Xbns1VdfpXfv3hx99NHcc889lYlh586dHHvssXTs2JEzzjiDk046CYgG\nzevRowdvvPFGyttIYw2JNFBzmo2HkktTV2AjPRnbt29fLrvsMoYMGVL5w6zZs2fz1FNPAXD22WdX\n/mp35syZLFy4kK9+9asAbNu2jY4dOwKwceNGWrZsuV91P/bYY4wcOZLrrruOefPmMXz4cJYuXUpO\nTg4nnXQSy5YtY/ny5YwYMYJvfvOb5OXl0aRJExYtWsSmTZu44IILWLp0aeWAc7uHo96dkFJFiUCk\ngSqw9Rk/GdsQPf/888yePZtnn32WH//4xyxZUnOy8zAo3E9/+tO9luXm5rJr1y5ycnL2GIoaoKys\nrHJE00T3339/5b2GTz75ZLZv38769esrkwtAr169aNGiBUuXLqW4uLhyfps2bRg4cCAvvPBCZSJI\n13DUSgQikh6tC1ObXFoX7vdLdu3axfvvv8/AgQPp378/06ZN49NPP2XAgAE8+uij3Hzzzfz5z3/m\n448/BmDQoEGcf/75TJgwgY4dO7Jx40a2bNlC165dOfLII3n33Xfp0aMHX/3qV1m5ciXvvfcenTt3\nZtq0aTz66KN71V9YWMjMmTMZOXIky5cvZ/v27eTn5/Pee+/RpUsXcnNz+de//sWKFSsoKiqivLyc\npk2b0qZNG7Zt28aMGTO44YYbKstL13DUSgQikh5Z6Ga65JJLmDVrFuvXr6egoIBbbrmFhx9+mM2b\nN+PujB8/njZt2nDbbbdxySWX0Lt3b0455RQKC6Mkc9RRR3HHHXcwePBgdu3aRdOmTfnNb35D165d\nOfvss5k1axY9evQgNzeXu+66i2984xvs3LmT0aNH07t3bwBuvfVWiouLOe+88/jFL37BFVdcweTJ\nkzEzpkyZgpkxZ84cJk2aRNOmTcnJyeHuu++mQ4cOLF68mBEjRrBz50527drFxRdfzDnnnAPAhx9+\nSPPmzTn00ENT3m6WeLa8viouLvYFCxZkOwyJkaKJz1M6KY3jv6dCSevUdw3Vobzly5fvcTOVxmbd\nunVcfvnlzJgxIyv1T548mVatWjFmzJhql1fX/ma20N2Lq31BAl01JCKShE6dOnHFFVdUXg6aaW3a\ntNnjPsappK4hEUkZD3fxaqwuvvjirNU9atSoGpfVtWdHiUCkGim/NDMNyrxDvbqhR15eHhs2bKB9\n+/aNOhnUN+7Ohg0byMvLO+AylAhEqpHySzPToP/E5ynNdhAJCgoKKCsrQ/cPyby8vDwKCg78a4ES\ngYikRNOmTenWrVu2w5ADoJPFIiIxp0QgIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc0oEIiIxp0Qg\nIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc2lLBGb2gJl9ZGZLE+a1M7MZZrYy/G2brvpFRCQ56Twi\nmAKcWWXeRGCmux8OzAzPRUQki9KWCNx9NrCxyuzzgYfC9EPAkHTVLyIiycn0OYJD3H1dmP4AOKSm\nFc1snJktMLMFutGFiEj6ZO1ksUc32azxRpvufq+7F7t7cX5+fgYjExGJl0wngg/NrBNA+PtRhusX\nEZEqMp0IpgMjwvQI4JkM1y8iIlWk8/LRx4B5wJFmVmZmY4BJwBlmthL4enguIiJZlLab17v7JTUs\nGpSuOiXGJh8Nm1enrLgy70BBykoTqd/SlghEMmrzaijZnLLi+k98ntKUlSZSv2mICRGRmFMiEBGJ\nOSUCEZGYUyIQEYk5JQIRkZhTIhARiTklAhGRmFMiEBGJOSUCEZGYUyIQEYk5JQIRkZhTIhARiTkl\nAhGRmFMiEBGJOSUCEZGYUyIQEYk5JQIRkZhTIhARiTndqlKkgercpjlFE59PWXmleSkrShoYJQKR\nBmruxNNTW2BJaouThkNdQyIiMadEICISc1lJBGY2wcyWmdlSM3vMzNQ7KSKSJRlPBGbWGRgPFLt7\nH6AJMCzTcYiISCRbXUO5QHMzywUOBtZmKQ4RkdjLeCJw9zXAz4HVwDpgs7v/tep6ZjbOzBaY2YLy\n8vJMhykiEhvZ6BpqC5wPdAMOA75iZt+uup673+vuxe5enJ+fn+kwRURiIxtdQ18H3nP3cnffATwF\nnJKFOEREhCQSgZldbWatLHK/mb1mZoPrUOdq4N/N7GAzM2AQsLwO5YmISB0kc0Qw2t0/AQYDbYHh\nwKQDrdDdXwWeAF4DloQY7j3Q8kREpG6SGWLCwt+zgIfdfVn4Jn/A3P024La6lCEiIqmRzBHBQjP7\nK1Ei+IuZtQR2pTcsERHJlGSOCMYAxwLvuvtWM2sPjEpvWCIikin7TATuvsvMPgSOCj8AE5FGqMw7\nUFDSOnUFti6ECUtSV56kzT537Gb2X8C3gDeBnWG2A7PTGJeIZFj/z++kdNLZqSswlUlF0iqZb/hD\ngCPd/fN0ByMiIpmXzMnid4Gm6Q5ERESyI5kjgq3AIjObCVQeFbj7+LRFJSIiGZNMIpgeHiIi0ggl\nc9XQQ5kIREREsiOZq4beI7pKaA/u3j0tEYmISEYl0zVUnDCdB1wEtEtPOCIikmn7vGrI3TckPNa4\n+y+BFF5sLCIi2ZRM19DxCU9ziI4Q9AtjEZFGIpkd+i8SpiuAUuDitEQjIiIZl8xVQwMzEYiIiGRH\nrecIzKyJmXVIeH5QuKm87igmItJI1JgIzGwYsBFYbGZ/D7enfBf4JnBZhuITEZE0q61r6GbgBHdf\nFU4YzwOGuvuzmQlNREQyobauoS/cfRWAu78GrFQSEBFpfGo7IuhoZtcmPG+T+Nzd/yd9YYmISKbU\nlgj+F2hZy3MREWkEakwE7n57JgOReOk36W+s2bQtZeWV5kHRxOdTVl7nNs1TVlZD0blN85S2YWle\nyoqSNNMvhCUr1mzaluLbIpLa8mJo7sTTU1tgSWqLk/RJ5g5lIiLSiO0zEZhZt2Tm7Q8za2NmT5jZ\nCjNbbmYn16U8ERE5cMkcETxZzbwn6ljvr4AX3L0ncAygXyqLiGRJjecIzKwn0BtobWYXJixqRXRf\nggNiZq2BAcBIAHf/AvjiQMsTEZG6qe1k8ZHAOUAb4NyE+VuAK+pQZzegHHjQzI4BFgJXu/tniSuZ\n2ThgHEBhYWEdqhMRkdrUdvnoM8AzZnayu89LcZ3HA99191fN7FfAROCWKvXfC9wLUFxcvNetMkVE\nJDWSOUdwpZm12f3EzNqa2QN1qLMMKHP3V8PzJ4gSg4iIZEEyiaCvu2/a/cTdPwaOO9AK3f0D4H0z\nOzLMGgS8eaDliYhI3STzg7IcM2sbEgBm1i7J19Xmu8AjZnYQ0dDWo+pYnoiIHKBkb1X5ipn9ITy/\nCPhxXSp190VE9z4WEZEsS+ZWlb83swXA7t+fX+ju6soRkX1K5dhFEI2HlPKhMKTW3xHkAVcCPYAl\nwD3uXpGpwESk4Uv1+E+pTiwSqe2I4CFgB/Ay0e0pewHXZCIoafzmNBsPJZemrsDW+q2JyIGqLREc\n5e5HA5jZ/cD8zIQkcVBg66Fkc7bDEBFqv3x0x+4JdQmJiDRetR0RHGNmn4RpA5qH5wa4u7dKe3Qi\nIpJ2tQ0x0SSTgYiISHboxjQiIjGnRCAiEnNKBCIiMadEICISc3UdPE5EpHqtC6GkdUqLnNOsA5Da\nXyuLEoGIpMuEJSkvsiDFiUUi6hoSEYk5JQIRkZhTIhARiTklAhGRmFMiEBGJOSUCEZGYUyIQEYk5\nJQIRkZhTIhARiTklAhGRmMvaEBNm1gRYAKxx93OyFYfsW79Jf2PNpm0pLbM0L6XFSYwUTXw+ZWV1\nbtOcuRNPT1l5DVU2xxq6GlgO6JaX9dyaTdsonZTigb5KUlucxEcqP4upTCoNWVa6hsysgGgIwfuy\nUb+IiHwpW+cIfglcD+zKUv0iIhJkPBGY2TnAR+6+cB/rjTOzBWa2oLy8PEPRiYjETzaOCPoB55lZ\nKTANON3MplZdyd3vdfdidy/Oz8/PdIwiIrGR8UTg7je6e4G7FwHDgL+5+7czHYeIiET0OwIRkZjL\n6q0q3X0WMCubMYiIxJ2OCEREYk43r5d9mtNsPJRcmtpCWxemtjwROWBKBLJPBbYeSjZnOwwRSRN1\nDYmIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGI\niMScEoGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJwSgYhIzCkRiIjE\nXMYTgZl1MbOXzOxNM1tmZldnOgYREflSbhbqrACuc/fXzKwlsNDMZrj7m1mIpVHqN+lvrNm0LWXl\nlealrCgRqYcyngjcfR2wLkxvMbPlQGdAiSBF1mzaRumks1NXYEnqihKR+ier5wjMrAg4Dni1mmXj\nzGyBmS0oLy/PdGgiIrGRtURgZi2AJ4Fr3P2Tqsvd/V53L3b34vz8/MwHKCISE9k4R4CZNSVKAo+4\n+1PZiKExm9NsPJRcmroCWxemriwRqXcyngjMzID7geXu/j+Zrj8OCmw9lGzOdhgi0kBko2uoHzAc\nON3MFoXHWVmIQ0REyM5VQ3MAy3S9IiJSPf2yWEQk5pQIRERiTolARCTmlAhERGJOiUBEJOay8oMy\nEZED0roQSlqnrLg5zToAKRyXq4FSIhCRhmPCkpQWV5DCpNKQqWtIRCTmlAhERGJOiUBEJOaUCERE\nYk6JQEQk5pQIRERiTpeP1geTj4bNq1NWXJl3oCBlpYlIY6dEUB9sXp3SG8n0n/g8pSkrTUQaOyWC\neqJo4vMpK6tzm+YpK0ukMVtHPp1S+KOyqLxVKSsvU5QI6onSSfqZu0impXqnncqkkkk6WSwiEnNK\nBCIiMadEICISczpHsL9SfKnvpO7NAAAI7ElEQVQn6HJPEckuJYL9leJLPUGXe4pIdikRiIikSJl3\nSP09DloXpvw+DFUpEYiIpEj/z+9M/aXgGbgkNSuJwMzOBH4FNAHuc/dJaass1X36rQtTV5aISD2Q\n8URgZk2A3wBnAGXAP81suru/mZYK09CnLyLSmGTj8tETgVXu/q67fwFMA87PQhwiIgKYu2e2QrOh\nwJnuPjY8Hw6c5O5XVVlvHDAuPD0SeCujgWZfB2B9toOop9Q2tVP71CxubdPV3fP3tVK9PVns7vcC\n92Y7jmwxswXuXpztOOojtU3t1D41U9tULxtdQ2uALgnPC8I8ERHJgmwkgn8Ch5tZNzM7CBgGTM9C\nHCIiQha6hty9wsyuAv5CdPnoA+6+LNNxNACx7RZLgtqmdmqfmqltqpHxk8UiIlK/aPRREZGYUyIQ\nEYk5JYIMM7MzzewtM1tlZhOrWT7ZzBaFx9tmtilh2c6EZY3yBHsS7VNoZi+Z2etmttjMzkpYdmN4\n3Vtm9o3MRp5+B9o2ZlZkZtsSPjv3ZD769Euifbqa2czQNrPMrCBh2QgzWxkeIzIbeT3g7npk6EF0\ncvwdoDtwEPAGcFQt63+X6GT67uefZnsbst0+RCf7/jNMHwWUJky/ATQDuoVymmR7m+pJ2xQBS7O9\nDfWgff4PGBGmTwceDtPtgHfD37Zhum22tymTDx0RZNb+Dq9xCfBYRiKrH5JpHwdahenWwNowfT4w\nzd0/d/f3gFWhvMaiLm0TB8m0z1HA38L0SwnLvwHMcPeN7v4xMAM4MwMx1xtKBJnVGXg/4XlZmLcX\nM+tK9M32bwmz88xsgZm9YmZD0hdm1iTTPiXAt82sDPgT0VFTsq9tyOrSNgDdQpfR383s1LRGmh3J\ntM8bwIVh+gKgpZm1T/K1jZoSQf01DHjC3XcmzOvq0c/jLwV+aWb/lp3QsuoSYIq7FwBnAQ+bmT7H\nkZraZh1Q6O7HAdcCj5pZq1rKaay+B5xmZq8DpxGNaLCz9pfEg/6BMmt/htcYRpVuIXdfE/6+C8wC\njkt9iFmVTPuMAf4A4O7zgDyigcQa+9AlB9w2obtsQ5i/kKgv/Yi0R5xZ+2wfd1/r7heGhPiDMG9T\nMq9t7JQIMiup4TXMrCfRSat5CfPamlmzMN0B6Aek5x4O2ZNM+6wGBgGYWS+inV15WG+YmTUzs27A\n4cD8jEWefgfcNmaWH+4Dgpl1J2qbdzMWeWbss33MrEPC0eONwANh+i/A4PA/1hYYHObFR7bPVsft\nQXTI/jbRt7IfhHk/BM5LWKcEmFTldacAS4j6OZcAY7K9LdloH6ITfnNDOywCBie89gfhdW8B38z2\nttSXtgH+H7AszHsNODfb25Kl9hkKrAzr3Ac0S3jtaKILDFYBo7K9LZl+aIgJEZGYU9eQiEjMKRGI\niMScEoGISMwpEYiIxJwSgYhIzCkRSNqEUS+Xpqisa8zs4H2sM9LMDktFfakQ4rmryrxRCaOAfmFm\nS8L0pP0su52ZXbmPdYaamZtZj4R5Xzezp6usN3X3kCVm1tTMfhZG8HzNzP7RGEdylT0pEUhDcQ1Q\nayIARgLVJoLdP6jKNnd/0N2PdfdjiQaFGxie7zVs8j60A2pNBERDTswJf5P1U6Jfah/l7scTjc3T\ncj9jkwZGiUDSLdfMHjGz5Wb2xO5v9WY2KAyCtsTMHkj41fRe881sPNEO/qUw3n4TM5tiZkvDehPM\nbChQDDwSvmE3N7NSM/svM3sNuMjMrjCzf5rZG2b2ZEIsU8zsnjCg39tmdk6YP9LMnglj1680s9t2\nb5SZfdvM5oe6fpfwy91RoYz5RL/+TpqZtQixzA9tcG6Yf3SIe5FFY+l3ByYBR9Z0NBHGEjoJuILo\nV7bJ1N+SKJmO92gET9z9A3d/Yn+2QxqgbP+iTY/G+yAaB9+BfuH5A0QDf+URjfZ4RJj/e6Jv/NXO\nD9OlROPmAJxANGzw7nrahL+zgOKE+aXA9QnP2ydM3wF8N0xPAV4g+mJ0ONHok3lEO8V1QHugObCU\nKNn0Ap4FmobX3w1cDnQiGuYhn2hM/LnAXbW0T+U2hec/A4aF6bZEv4DNA34LfCvMbxbm9QAW1VL2\nCOB3YXo+cEyY/jrwdJV1pwJDgOOBf2b7c6NH5h86IpB0e9/d54bpqUB/4EjgPXd/O8x/CBhQy/yq\n3gW6m9mvzexM4JNa6n88YbqPmb1sZkuAy4DeCcv+4O673H1lKL9nmD/D3Te4+zbgqRD/IKJk9E8z\nWxSedyf6Bj7L3cs9+kadWHcyBgM/CGW+RLTDLwT+AdxsZtcDXdx9exJlXUI0Jj/h7+7uoZqGEtAQ\nAzGWm+0ApNGruoOp8w7H3T82s2OIbihyJXAx0Vgx1fksYXoKMMTd3zCzkcDXkoizuvkGPOTuNyYu\nsLrfI8JCfO9Umf+2mc0DzgZeMLPR1HLTGTPLJxpmuZeZOdH/+Q4zuxHYQHS0kagdsJ5oHJ5uZtbC\n3T+t47ZIA6IjAkm3QjM7OUxfSnTy8i2gKOFqluHA32uZD7CFcNLSotFXc9z9SeBmoi6NPdapQUtg\nnZk1JToiSHSRmeVYdI+H7iEWgDPCFTrNibpP5gIzgaFm1jHE086iGwm9SjTefftQx0VJtE+iv5Bw\nMxkzOy787e7uq9z9V8BzQN99bOtFRLc47eruRR7dn2AtcDKwAuhqZkeEsrsRHRktdvctRN1xvwzx\nY2Ydw/kXacSUCCTd3gK+Y2bLib6J/jZ0bYwC/i900+wC7qlpfijnXqJvwy8R3T1qVuhCmUo0pDBE\n3/jv2X2yuJpYbiHaWc8l2iEmWk3Ul/5n4MqE7pf5wJPAYuBJd1/g7m8SJaC/mtliolsbdnL3dUQj\nx84LdSzfz7a6HfhKOAG+LJQFcKmZLQvbewQw1d0/BBaGdaueLL4E+GOVeU8Cl4TtupzopjWLiLqv\nRockADAR2AQsD+/BdGDzfm6HNDAafVRiz8ymAM95latjQvdRsbtflY24RDJFRwQiIjGnIwIRkZjT\nEYGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjM/X80m6Twjj4fNQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r = [min(min(auc_max), min(auc_1sde)), max(max(auc_max), max(auc_1sde))]\n",
    "b = 15\n",
    "fig = plt.figure()\n",
    "frame = plt.gca()\n",
    "ax = fig.add_subplot(111)\n",
    "h1 = plt.hist(auc_max, range=r, bins=b, normed=True, histtype='step', stacked=True,\n",
    "              label='max({})'.format(np.round(np.mean(auc_max),3)))\n",
    "\n",
    "h2 = plt.hist(auc_1sde,range=r, bins=b, normed=True, histtype='step', stacked=True,\n",
    "              label='1sde({})'.format(np.round(np.mean(auc_1sde),3)))\n",
    "plt.legend()\n",
    "\n",
    "ax.set_xlabel('bootstrapped Test AUC')\n",
    "ax.set_ylabel('Pct Runs')\n",
    "plt.title('Distribution of BootStrap Test AUC')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
